Bad Science Part 12

Science coverage is further crippled, of course, by the fact that the subject can be quite difficult to understand. This in itself can seem like an insult to intelligent people, like journalists, who fancy themselves as able to understand most things, but there has also been an acceleration in complexity in recent times. Fifty years ago you could sketch out a full explanation of how an AM radio worked on the back of a napkin, using basic school-level knowledge of science, and build a crystal set in a cla.s.sroom which was essentially the same as the one in your car. When your parents were young they could fix their own car, and understand the science behind most of the everyday technology they encountered, but this is no longer the case. Even a geek today would struggle to give an explanation of how his mobile phone works, because technology has become more difficult to understand and explain, and everyday gadgets have taken on a 'black box' complexity that can feel sinister, as well as intellectually undermining. The seeds were sown.

But we should return to the point. If there was little science, then what did did appear in all these long stories on MMR? Going back to the 2002 data from the ESRC, only a quarter mentioned Andrew Wakefield, which seems odd, considering he was the cornerstone of the story. This created the erroneous impression that there was a large body of medical opinion which was suspicious of MMR, rather than just one 'maverick'. Less than a third of broadsheet reports referred to the overwhelming evidence that MMR is safe, and only 11 per cent mentioned that it is regarded as safe in the ninety other countries in which it is used. appear in all these long stories on MMR? Going back to the 2002 data from the ESRC, only a quarter mentioned Andrew Wakefield, which seems odd, considering he was the cornerstone of the story. This created the erroneous impression that there was a large body of medical opinion which was suspicious of MMR, rather than just one 'maverick'. Less than a third of broadsheet reports referred to the overwhelming evidence that MMR is safe, and only 11 per cent mentioned that it is regarded as safe in the ninety other countries in which it is used.

It was rare to find much discussion of the evidence at all, as it was considered too complicated, and when doctors tried to explain it they were frequently shouted down, or worse still, their explanations were condensed into bland statements that 'science had shown' there was nothing to worry about. This uninformative dismissal was pitted against the emotive concerns of distressed parents.

As 2002 wore on, things got really strange. Some newspapers, such as the Daily Mail Daily Mail and the and the Daily Telegraph Daily Telegraph, made MMR the focus of a ma.s.sive political campaign, and the beatification of Wakefield reached a kind of fever pitch. Lorraine Fraser had an exclusive interview with him in the Telegraph Telegraph in which he was described as 'a champion of patients who feel their fears have been ignored'. She wrote a dozen similar articles over the next year (and her reward came when she was named British Press Awards Health Writer of the Year 2002, a gong I do not myself expect to receive). in which he was described as 'a champion of patients who feel their fears have been ignored'. She wrote a dozen similar articles over the next year (and her reward came when she was named British Press Awards Health Writer of the Year 2002, a gong I do not myself expect to receive).

Justine Picardie did a lavish photo feature on Wakefield, his house and his family for the Telegraph Telegraph Sat.u.r.day magazine. Andy is, she tells us, 'a handsome, glossy-haired hero to families of autistic children'. How are the family? 'A likeable, lively family, the kind you would be happy to have as friends, pitted against mysterious forces who have planted bugging devices and have stolen patients' records in 'apparently inexplicable' burglaries.' She fantasises-and I absolutely promise you I'm not making this up-about a Hollywood depiction of Wakefield's heroic struggle, with Russell Crowe playing the lead 'opposite Julia Roberts as a feisty single mother fighting for justice for her child'. Sat.u.r.day magazine. Andy is, she tells us, 'a handsome, glossy-haired hero to families of autistic children'. How are the family? 'A likeable, lively family, the kind you would be happy to have as friends, pitted against mysterious forces who have planted bugging devices and have stolen patients' records in 'apparently inexplicable' burglaries.' She fantasises-and I absolutely promise you I'm not making this up-about a Hollywood depiction of Wakefield's heroic struggle, with Russell Crowe playing the lead 'opposite Julia Roberts as a feisty single mother fighting for justice for her child'.

The evidence on MMR So what is the evidence on the safety of MMR?

There are a number of ways to approach the evidence on the safety of a given intervention, depending on how much attention you have to give. The simplest approach is to pick an arbitrary authority figure: a doctor, perhaps, although this seems not to be appealing (in surveys people say they trust doctors the most, and journalists the least: this shows the flaw in that kind of survey).

You could take another, larger authority at face value, if there is one that suits you. The Inst.i.tute of Medicine, the Royal Colleges, the NHS, and more, all came out in support of MMR, but this was apparently not sufficient to convince. You could offer information: an NHS website at mmrthefacts.nhs.uk mmrthefacts.nhs.uk started with the phrase 'MMR is safe' (literally), and allowed the reader to drill down to the detail of individual studies.* started with the phrase 'MMR is safe' (literally), and allowed the reader to drill down to the detail of individual studies.*

- Whether you buy the DoH phrase 'MMR is safe' depends on what you decide you mean by 'safe'. Is flying safe? Is your was.h.i.+ng machine safe? What are you sitting on? Is that safe? You can obsess over the idea that philosophically nothing can ever be shown to be 100 per cent safe-and many will-but you would be arguing about a fairly meaningless and uncommon definition of the word. - Whether you buy the DoH phrase 'MMR is safe' depends on what you decide you mean by 'safe'. Is flying safe? Is your was.h.i.+ng machine safe? What are you sitting on? Is that safe? You can obsess over the idea that philosophically nothing can ever be shown to be 100 per cent safe-and many will-but you would be arguing about a fairly meaningless and uncommon definition of the word.

But that too did little to stem the tide. Once a scare is running, perhaps every refutation can seem like an admission of guilt, drawing attention to the scare.

The Cochrane Collaboration is as blemishless as they come, and it has done a systematic review of the literature on MMR, concluding that there was no evidence that it is unsafe (although it didn't appear until 2005). This reviewed the data the media had systematically ignored: what was in it?

If we are to maintain the moral high ground, there are a few things we need to understand about evidence. Firstly, there is no single golden study which proves that MMR is safe (although the evidence to say it is dangerous was exceptionally poor). There is, for example, no randomised controlled trial. We are presented instead with a huge mora.s.s of data, from a number of different studies, all of which are flawed in their own idiosyncratic ways for reasons of cost, competence and so on. A common problem with applying old data to new questions is that these papers and datasets might have a lot of useful information, which was collected very competently to answer the questions which the researchers were interested in at the time, but which isn't perfect for your needs. It's just, perhaps, pretty good.

Smeeth et al et al., for example, did something called a 'case-control' study, using the GP Research Database. This is a common type of study, where you take a bunch of people with the condition you're looking at ('autism'), and a bunch of people without it, then look to see if there is any difference in how much each group was exposed to the thing you think might be causing the condition ('MMR').

If you care who paid for the study-and I hope you've become a bit more sophisticated than that by now-it was funded by the Medical Research Council. They found around 1,300 people with autism, and then got some 'controls': random people who did not have autism, but with the same age, s.e.x, and general practice. Then they looked to see if vaccination was any more common in the people with autism, or the controls, and found no difference between the two groups. The same researchers also did a systematic review of similar studies in the United States and Scandinavia, and again, pooling the data, found no link between MMR and autism.

There is a practical problem with this kind of research, of course, which I would hope you might spot: most people do do get the MMR vaccine, so the individuals you're measuring who get the MMR vaccine, so the individuals you're measuring who didn't didn't get the vaccine might be unusual in other ways-perhaps their parents have refused the vaccine for ideological or cultural reasons, or the child has a pre-existing physical health problem-and those factors might themselves be related to autism. There's little you can do in terms of study design about this potential 'confounding variable', because as we said, you're not likely to do a randomised controlled trial in which you randomly don't give children vaccines: you just throw the result into the pot with the rest of the information, in order to reach your verdict. As it happens, Smeeth get the vaccine might be unusual in other ways-perhaps their parents have refused the vaccine for ideological or cultural reasons, or the child has a pre-existing physical health problem-and those factors might themselves be related to autism. There's little you can do in terms of study design about this potential 'confounding variable', because as we said, you're not likely to do a randomised controlled trial in which you randomly don't give children vaccines: you just throw the result into the pot with the rest of the information, in order to reach your verdict. As it happens, Smeeth et al et al. went to great lengths to make sure their controls were representative. If you like, you can read the paper and decide if you agree.

So 'Smeeth' was a 'case-control study', where you compare groups which had the outcome or not, and look at how common the exposure was in each group. In Denmark, Madsen et al et al. did the opposite kind of study, called a 'cohort study': you compare groups which had the exposure or not, in order to see whether there is any variation in the outcome. In this specific case, then, you take two groups of people, who either had MMR or didn't, and then check later to see if the rate of autism is any different between the two groups.

This study was big-very big-and included all the children born in Denmark between January 1991 and December 1998. In Denmark there is a system of unique personal identification numbers, linked to vaccination registers and information about the diagnosis of autism, which made it possible to chase up almost all the children in the study. This was a pretty impressive achievement, since there were 440,655 children who were vaccinated, and 96,648 who were unvaccinated. No difference was found between vaccinated and unvaccinated children, in the rates of autism or autistic spectrum disorders, and no a.s.sociation between development of autism and age at vaccination.

Anti-MMR campaigners have responded to this work by saying that only a small number of children are harmed by the vaccine, which seems to be inconsistent with their claims that MMR is responsible for a ma.s.sive upswing in diagnoses of autism. In any case, if a vaccine caused an adverse reaction in a very small number of people, that would be no surprise-it would be no different from any other medical intervention (or, arguably, any human activity), and there would be, surely, no story.

As with all studies, there are problems with this huge study. The follow-up of diagnostic records ends one year (31 December 1999) after the last day of admission to the cohort: so, because autism comes on after the age of one year, the children born later in the cohort would be unlikely to have shown up with autism by the end of the follow-up period. But this is nagged up in the study, and you can decide whether you think it undermines its overall findings. I don't think it's much of a problem. That's my verdict, and I think you might agree that it's not a particularly foolish one. It did run from January 1991, after all.

This is the kind of evidence you will find in the Cochrane review, which found, very simply, that 'existing evidence on the safety and effectiveness of MMR vaccine supports current policies of ma.s.s immunisation aimed at global measles eradication in order to reduce morbidity and mortality a.s.sociated with mumps and rubella'.

It also contained multiple criticisms of the evidence it reviewed, which, bizarrely, has been seized upon by various commentators to claim that there was some kind of st.i.tch-up. The review was heading towards a conclusion that MMR was risky, they say, if you read the content, but then, out of nowhere, it produced a rea.s.suring conclusion, doubtless because of hidden political pressure.

The Daily Mail's Daily Mail's Melanie Phillips, a leading light of the anti-vaccination movement, was outraged by what she thought she had found: 'It said that no fewer than nine of the most celebrated studies that have been used against [Andrew Wakefield] were unreliable in the way they were constructed.' Of course it did. I'm amazed it wasn't more. Cochrane reviews are Melanie Phillips, a leading light of the anti-vaccination movement, was outraged by what she thought she had found: 'It said that no fewer than nine of the most celebrated studies that have been used against [Andrew Wakefield] were unreliable in the way they were constructed.' Of course it did. I'm amazed it wasn't more. Cochrane reviews are intended intended to criticise papers. to criticise papers.

Scientific 'evidence' in the media But the newspapers in 2002 had more than just worried parents. There was a smattering of science to keep things going: you will remember computer-generated imagery of viruses and gut walls, perhaps, and stories about laboratory findings. Why have I not mentioned those?

For one thing, these important scientific findings were being reported in newspapers and magazines, and at meetings, in fact anywhere except proper academic journals where they could be read and carefully appraised. In May, for example, Wakefield 'exclusively revealed' that 'more than 95 per cent of those who had the virus in their gut had MMR as their only doc.u.mented exposure to measles'. He doesn't appear to have revealed this in a peer-reviewed academic journal, but in a weekend colour supplement.

Other people started popping up all over the place, claiming to have made some great finding, but never publis.h.i.+ng their research in proper, peer-reviewed academic journals. A pharmacist in Sunderland called Mr Paul Shattock was reported on the Today Today programme, and in several national newspapers, to have identified a distinct subgroup of children with autism resulting from MMR. Mr Shattock is very active on anti-immunisation websites, but he still doesn't seem to have got round to publis.h.i.+ng this important work years later, even though the Medical Research Council suggested in 2002 that he should 'publish his research and come forward to the MRC with positive proposals'. programme, and in several national newspapers, to have identified a distinct subgroup of children with autism resulting from MMR. Mr Shattock is very active on anti-immunisation websites, but he still doesn't seem to have got round to publis.h.i.+ng this important work years later, even though the Medical Research Council suggested in 2002 that he should 'publish his research and come forward to the MRC with positive proposals'.

Meanwhile Dr Arthur Krigsman, paediatric gastrointestinal consultant working in the New York area, was telling hearings in Was.h.i.+ngton that he had made all kinds of interesting findings in the bowels of autistic children, using endoscopes. This was lavishly reported in the media. Here is the Daily Telegraph Daily Telegraph: Scientists in America have reported the first independent corroboration of the research findings of Dr Andrew Wakefield. Dr Krigsman's discovery is significant because it independently supports Dr Wakefield's conclusion that a previously unidentified and devastating combination of bowel and brain disease is afflicting young children-a claim that the Department of Health has dismissed as 'bad science'. Scientists in America have reported the first independent corroboration of the research findings of Dr Andrew Wakefield. Dr Krigsman's discovery is significant because it independently supports Dr Wakefield's conclusion that a previously unidentified and devastating combination of bowel and brain disease is afflicting young children-a claim that the Department of Health has dismissed as 'bad science'.

To the best of my knowledge-and I'm pretty good at searching for this stuff- Krigsman's new scientific research findings which corroborate Andrew Wakefield's have never been published in an academic journal: certainly there is no trace of them on Pubmed, the index of nearly all medical academic articles.

In case the reason why this is important has not sunk in, let me explain again. If you visit the premises of the Royal Society in London, you'll see its motto proudly on display: 'Nullius in verba '-'On the word of no one'. What I like to imagine this refers to, in my geeky way, is the importance of publis.h.i.+ng proper scientific papers if you want people to pay attention to your work. Dr Arthur Krigsman has been claiming for years now that he has found evidence linking MMR to autism and bowel disease. Since he hasn't published his findings, he can claim them until he's blue in the face, because until we can see exactly what he did, we can't see what flaws there may be in his methods. Maybe he didn't select the subjects properly. Maybe he measured the wrong things. If he doesn't write it up formally, we can never know, because that is what scientists do: write papers, and pull them apart to see if their findings are robust. '-'On the word of no one'. What I like to imagine this refers to, in my geeky way, is the importance of publis.h.i.+ng proper scientific papers if you want people to pay attention to your work. Dr Arthur Krigsman has been claiming for years now that he has found evidence linking MMR to autism and bowel disease. Since he hasn't published his findings, he can claim them until he's blue in the face, because until we can see exactly what he did, we can't see what flaws there may be in his methods. Maybe he didn't select the subjects properly. Maybe he measured the wrong things. If he doesn't write it up formally, we can never know, because that is what scientists do: write papers, and pull them apart to see if their findings are robust.

Krigsman and others' failures to publish in peer-reviewed academic journals weren't isolated incidents. In fact it's still going on, years later. In 2006, exactly the same thing was happening again. 'US Scientists Back Autism Link to MMR', squealed the Telegraph Telegraph. 'Scientists Fear MMR Link to Autism', roared the Mail Mail. 'US Study Supports Claims of MMR Link to Autism', croaked The Times The Times a day later. a day later.

What was this frightening new data? These scare stories were based on a poster presentation, at a conference yet to occur, on research not yet completed, by a man with a track record of announcing research that never subsequently appears in an academic journal. In fact, astonis.h.i.+ngly, four years later, it was Dr Arthur Krigsman again. The story this time was different: he had found genetic material (RNA) from vaccine-strain measles virus in some gut samples from children with autism and bowel problems. If true, this would fit with Wakefield's theory, which by 2006 was lying in tatters. We might also mention that Wakefield and Krigsman are doctors together at Thoughtful House, a private autism clinic in the USA offering eccentric treatments for developmental disorders.

The Telegraph Telegraph went on to explain that Krigsman's most recent unpublished claim was replicating similar work from 1998 by Dr Andrew Wakefield, and from 2002 by Professor John O'Leary. This was, to say the least, a mis-statement. There is no work from 1998 by Wakefield which fits the went on to explain that Krigsman's most recent unpublished claim was replicating similar work from 1998 by Dr Andrew Wakefield, and from 2002 by Professor John O'Leary. This was, to say the least, a mis-statement. There is no work from 1998 by Wakefield which fits the Telegraph's Telegraph's claim-at least not in PubMed that I can find. I suspect the newspaper was confused about the infamous claim-at least not in PubMed that I can find. I suspect the newspaper was confused about the infamous Lancet Lancet paper on MMR, which by 2004 had already been partially retracted. paper on MMR, which by 2004 had already been partially retracted.

There are, however, two papers suggesting that traces of genetic material from the measles virus have been found in children. They have received a mountain of media coverage over half a decade, and yet the media have remained studiously silent on the published evidence suggesting that they were false positives, as we will now see.

One is from Kawas.h.i.+ma et al et al. in 2002, also featuring Wakefield as an author, in which it is claimed that genetic material from measles vaccine was found in blood cells. Doubt is cast on this both by attempts to replicate it, showing where the false positives probably appeared, and by the testimony of Nick Chadwick, the PhD student whose work we described above. Even Andrew Wakefield himself no longer relies on this paper.

The other is O'Leary's paper from 2002, also featuring Wakefield as an author, which produced evidence of measles RNA in tissue samples from children. Futher experiments, again, have ill.u.s.trated where the false positives seem to have arisen, and in 2004, when Professor Stephen Bustin was examining the evidence for the legal aid case, he explained how he established to his satisfaction-during a visit to the O'Leary lab-that these were false positives due to contamination and inadequate experimental methods. He has shown, firstly, that there were no 'controls' to check for false positives (contamination is a huge risk when you are looking for minuscule traces of genetic material, so you generally run 'blank' samples to make sure they do come out blank); he found calibration problems with the machines; problems with log books; and worse. He expanded on this at enormous length in a US court case on autism and vaccines in 2006. You can read his detailed explanation in full online. To my astonishment not one journalist in the UK has ever bothered to report it.

Both of these papers claiming to show a link received blanket media coverage at the time, as did Krigsman's claims.

What they didn't tell you In the May 2006 issue of the Journal of Medical Virology Journal of Medical Virology there was a very similar study to the one described by Krigsman, only this one had actually been published, by Afzal there was a very similar study to the one described by Krigsman, only this one had actually been published, by Afzal et al et al. It looked for measles RNA in children with regressive autism after MMR vaccination, much like the unpublished Krigsman study, and it used tools so powerful they could detect measles RNA down to single-figure copy numbers. It found no evidence of the magic vaccine-strain measles RNA to implicate MMR. Perhaps because of that unfrightening result, the study was loudly ignored by the press.

Because it has been published in full, I can read it, and pick holes in it, and I am more than happy to do so: because science is about critiquing openly published data and methodologies, rather than press-released chimeras, and in the real world all studies have some flaws, to a greater or lesser extent. Often they are practical ones: here, for example, the researchers couldn't get hold of the tissue they ideally would have used, because they could not get ethics committee approval for intrusive procedures like lumbar punctures and gut biopsies on children (Wakefield did manage to obtain such samples, but he is, we should remember, currently going through a GMC professional conduct hearing over the issue).

Surely they could have borrowed some existing samples, from children said to be damaged by vaccines? You'd have thought so. They report in the paper that they tried to ask anti-MMR researchers-if that's not an unfair term-whether they could borrow some of their tissue samples to work on. They were ignored.*

- 'The groups of investigators that either had access to original autism specimens or investigated them later for measles virus detection were invited to take part in the study but failed to respond. Similarly, it was not possible to obtain clinical specimens of autism cases from these investigators for independent investigations.' - 'The groups of investigators that either had access to original autism specimens or investigated them later for measles virus detection were invited to take part in the study but failed to respond. Similarly, it was not possible to obtain clinical specimens of autism cases from these investigators for independent investigations.'

Afzal et al et al. was not reported in the media, anywhere at all, except by me, in my column.

This is not an isolated case. Another major paper was published in the leading academic journal Pediatrics Pediatrics a few months later-to complete media silence-again suggesting very strongly that the earlier results from Kawas.h.i.+ma and O'Leary were in error, and false positives. D'Souza a few months later-to complete media silence-again suggesting very strongly that the earlier results from Kawas.h.i.+ma and O'Leary were in error, and false positives. D'Souza et al et al. replicated the earlier experiments very closely, and in some respects more carefully: most importantly, it traced out the possible routes by which a false positive could have occurred, and made some astonis.h.i.+ng findings.

False positives are common in PCR, because it works by using enzymes to replicate RNA, so you start with a small amount in your sample, which is then 'amplified up', copied over and over again, until you have enough to measure and work with. Beginning with a single molecule of genetic material, PCR can generate 100 billion similar molecules in an afternoon. Because of this, the PCR process is exquisitely sensitive to contamination-as numerous innocent people languis.h.i.+ng in jail could tell you-so you have to be very careful, and clean up as you go.

As well as raising concerns about contamination, D'Souza also found that the O'Leary method might have accidentally amplified the wrong bits of RNA.

Let's be clear: this is absolutely not about criticising individual researchers. Techniques move on, results are sometimes not replicable, and not all double-checking is practical (although Bustin's testimony is that standards in the O'Leary lab were problematic). But what is striking is that the media rabidly picked up on the original frightening data, and then completely ignored the new rea.s.suring data. This study by D'Souza, like Afzal before it, was unanimously ignored by the media. It was covered, by my count, in: my column; one Reuters piece which was picked up by n.o.body; and one post on the lead researcher's boyfriend's blog (where he talked about how proud he was of his girlfriend). Nowhere else.*

- In 2008, just as this chapter was being put to bed, some journalists deigned-miraculously-to cover a PCR experiment with a negative finding. It was misreported as the definitive refutation of the entire MMR-autism hypothesis. This was a childish overstatement, and that doesn't help anyone either. I am not hard to please. - In 2008, just as this chapter was being put to bed, some journalists deigned-miraculously-to cover a PCR experiment with a negative finding. It was misreported as the definitive refutation of the entire MMR-autism hypothesis. This was a childish overstatement, and that doesn't help anyone either. I am not hard to please.

You could say, very reasonably, that this is all very much par for the course: newspapers report the news, and it's not very interesting if a piece of research comes out saying something is safe. But I would argue-perhaps sanctimoniously-that the media have a special responsibility in this case, because they themselves demanded 'more research', and moreover because at the very same time at the very same time that they were ignoring properly conducted and fully published negative findings, they were talking up scary findings from an unpublished study by Krigsman, a man with a track record of making scary claims which remain unpublished. that they were ignoring properly conducted and fully published negative findings, they were talking up scary findings from an unpublished study by Krigsman, a man with a track record of making scary claims which remain unpublished.

MMR is not an isolated case in this regard. You might remember the scare stories about mercury fillings from the past two decades: they come around every few years, usually accompanied by a personal anecdote in which fatigue, dizziness and headaches are all vanquished following the removal of the fillings by one visionary dentist. Traditionally these stories conclude with a suggestion that the dental establishment may well be covering up the truth about mercury, and a demand for more research into its safety.

The first large-scale randomised control trials on the safety of mercury fillings were published recently, and if you were waiting to see these hotly antic.i.p.ated results, personally demanded by journalists on innumerable newspapers, you'd be out of luck, because they were reported nowhere. Nowhere. A study of more than 1,000 children, where some were given mercury fillings and some mercury-free fillings, measuring kidney function and neurodevelopmental outcomes like memory, coordination, nerve conduction, IQ and so on over several years. It was a well-conducted study. There were no significant differences between the two groups. That's worth knowing about if you've ever been scared by the media's reports on mercury fillings-and by G.o.d, you'd have been scared.

Panorama featured a particularly chilling doc.u.mentary in 1994 called featured a particularly chilling doc.u.mentary in 1994 called The Poison in Your Mouth The Poison in Your Mouth. It opened with dramatic footage of men in full protective gear rolling barrels of mercury around. I'm not giving you the definitive last word on mercury here. But I think we can safely a.s.sume there is no Panorama Panorama doc.u.mentary in the pipeline covering the startling new research data suggesting that mercury fillings may not be harmful after all. In some respects this is just one more ill.u.s.tration of how unreliable intuition can be in a.s.sessing risks like those presented with a vaccine: not only is it a flawed strategy for this kind of numerical a.s.sessment, on outcomes which are too rare for one person to collect meaningful data on them in their personal journey through life; but the information you are fed by the media about the wider population is ludicrously, outrageously, criminally crooked. So at the end of all this, what has the British news media establishment achieved? doc.u.mentary in the pipeline covering the startling new research data suggesting that mercury fillings may not be harmful after all. In some respects this is just one more ill.u.s.tration of how unreliable intuition can be in a.s.sessing risks like those presented with a vaccine: not only is it a flawed strategy for this kind of numerical a.s.sessment, on outcomes which are too rare for one person to collect meaningful data on them in their personal journey through life; but the information you are fed by the media about the wider population is ludicrously, outrageously, criminally crooked. So at the end of all this, what has the British news media establishment achieved?

Old diseases return It's hardly surprising that the MMR vaccination rate has fallen from 92 per cent in 1996 to 73 per cent today. In some parts of London it's down to 60 per cent, and figures from 2004-05 showed that in Westminster only 38 per cent of children had both jabs by the age of five.*

- Not 11.7 per cent as claimed in the - Not 11.7 per cent as claimed in the Telegraph Telegraph and the and the Daily Mail Daily Mail in February and June 2006. in February and June 2006.

It is difficult to imagine what could be driving this, if not a brilliantly successful and well-coordinated media anti-MMR campaign, which pitched emotion and hysteria against scientific evidence. Because people listen to journalists: this has been demonstrated repeatedly, and not just with the kinds of stories in this book.

A 2005 study in the Medical Journal of Australia Medical Journal of Australia looked at mammogram bookings, and found that during the peak media coverage of Kylie Minogue's breast cancer, bookings rose by 40 per cent. The increase among previously unscreened women in the forty-to-sixty-nine-year age group was 101 per cent. These surges were unprecedented. And I'm not cherry-picking: a systematic review from the Cochrane Collaboration found five studies looking at the use of specific health interventions before and after media coverage of specific stories, and each found that favourable publicity was a.s.sociated with greater use, and unfavourable coverage with lower use. looked at mammogram bookings, and found that during the peak media coverage of Kylie Minogue's breast cancer, bookings rose by 40 per cent. The increase among previously unscreened women in the forty-to-sixty-nine-year age group was 101 per cent. These surges were unprecedented. And I'm not cherry-picking: a systematic review from the Cochrane Collaboration found five studies looking at the use of specific health interventions before and after media coverage of specific stories, and each found that favourable publicity was a.s.sociated with greater use, and unfavourable coverage with lower use.

It's not just the public: medical practice is influenced by the media too, and so are academics. A mischievous paper from the New England Journal of Medicine New England Journal of Medicine in 1991 showed that if a study was covered by the in 1991 showed that if a study was covered by the New York Times New York Times, it was significantly more likely to be cited by other academic papers. Having come this far, you are probably unpicking this study already. Was coverage in the New York Times New York Times just a surrogate marker for the importance of the research? History provided the researchers with a control group to compare their results against: for three months, large parts of the paper went on strike, and while the journalists did produce an 'edition of record', this newspaper was never actually printed. They wrote stories about academic research, using the same criteria to judge importance that they always had, but the research they wrote about in articles which never saw the light of day saw no increase in citations. just a surrogate marker for the importance of the research? History provided the researchers with a control group to compare their results against: for three months, large parts of the paper went on strike, and while the journalists did produce an 'edition of record', this newspaper was never actually printed. They wrote stories about academic research, using the same criteria to judge importance that they always had, but the research they wrote about in articles which never saw the light of day saw no increase in citations.

People read newspapers. Despite everything we think we know, their contents seep in, we believe them to be true, and we act upon them, which makes it all the more tragic that their contents are so routinely flawed. Am I extrapolating unfairly from the extreme examples in this book? Perhaps not. In 2008 Gary Schwitzer, an ex-journalist who now works on quant.i.tative studies of the media, published an a.n.a.lysis of five hundred health articles covering treatments from mainstream newspapers in the US. Only 35 per cent of stories were rated satisfactory for whether the journalist had 'discussed the study methodology and the quality of the evidence' (because in the media, as we have seen repeatedly in this book, science is about absolute truth statements from arbitrary authority figures in white coats, rather than clear descriptions of studies, and the reasons why people draw conclusions from them). Only 28 per cent adequately covered benefits, and only 33 per cent adequately covered harms. Articles routinely failed to give any useful quant.i.tative information in absolute terms, preferring unhelpful eye-catchers like '50 per cent higher' instead.

In fact there have been systematic quant.i.tative surveys of the accuracy of health coverage in Canada, Australia and America-I'm trying to get one off the ground in the UK-and the results have been universally unimpressive. It seems to me that the state of health coverage in the UK could well be a serious public health issue.

Meanwhile, the incidence of two of the three diseases covered by MMR is now increasing very impressively. We have the highest number of measles cases in England and Wales since current surveillance methods began in 1995, with cases occurring mostly in children who had not been adequately vaccinated: 971 confirmed cases were reported in 2007 (mostly a.s.sociated with prolonged outbreaks in travelling and religious communities, where vaccine uptake has been historically low), after 740 cases in 2006 (and the first death since 1992). Seventy-three per cent of cases were in the SouthEast, and most of those were in London.

Mumps began rising again in 1999, after many years of cases in only double figures: by 2005 the United Kingdom had a mumps epidemic, with around 5,000 notifications in January alone.

A lot of people who campaign against vaccines like to pretend that they don't do much good, and that the diseases they protect against were never very serious anyway. I don't want to force anyone to have their child vaccinated, but equally I don't think anyone is helped by misleading information. By contrast with the unlikely event of autism being a.s.sociated with MMR, the risks from measles, though small, are real and quantifiable. The Peckham Report on immunisation policy, published shortly after the introduction of the MMR vaccine, surveyed the recent experience of measles in Western countries and estimated that for every 1,000 cases notified, there would be 0.2 deaths, ten hospital admissions, ten neurological complications and forty respiratory complications. These estimates have been borne out in recent minor epidemics in the Netherlands (1999: 2,300 cases in a community philosophically opposed to vaccination, three deaths), Ireland (2000: 1,200 cases, three deaths) and Italy (2002: three deaths). It's worth noting that plenty of these deaths were in previously healthy children, in developed countries, with good healthcare systems.

Though mumps is rarely fatal, it's an unpleasant disease with unpleasant complications (including meningitis, pancreat.i.tis and sterility). Congenital rubella syndrome has become increasingly rare since the introduction of MMR, but causes profound disabilities including deafness, autism, blindness and mental handicap, resulting from damage to the foetus during early pregnancy.

The other thing you will hear a lot is that vaccines don't make much difference anyway, because all the advances in health and life expectancy have been due to improvements in public health for a wide range of other reasons. As someone with a particular interest in epidemiology and public health, I find this suggestion flattering; and there is absolutely no doubt that deaths from measles began to fall over the whole of the past century for all kinds of reasons, many of them social and political as well as medical: better nutrition, better access to good medical care, antibiotics, less crowded living conditions, improved sanitation, and so on.

Life expectancy in general has soared over the past century, and it's easy to forget just how phenomenal this change has been. In 1901, males born in the UK could expect to live to forty-five, and females to forty-nine. By 2004, life expectancy at birth had risen to seventy-seven for men, and eighty-one for women (although of course much of the change is due to reductions in infant mortality).

So we are living longer, and vaccines are clearly not the only reason why. No single change is the reason why. Measles incidence dropped hugely over the preceding century, but you would have to work fairly hard to persuade yourself that vaccines had no impact on that. Here, for example, is a graph showing the reported incidence of measles from 1950 to 2000 in the United States.

For those who think that single vaccines for the components of MMR are a good idea, you'll notice that these have been around since the 1970s, but that a concerted programme of vaccination-and the concerted programme of giving all three vaccinations in one go as MMR-is fairly clearly a.s.sociated in time with a further (and actually rather definitive) drop in the rate of measles cases.

The same is true for mumps:

While we're thinking about mumps, let's not forget our epidemic in 2005, a resurgence of a disease many young doctors would struggle even to recognise. Here is a graph of mumps cases from the BMJ BMJ article that a.n.a.lysed the outbreak: article that a.n.a.lysed the outbreak:

Almost all confirmed cases during this outbreak were in people aged fifteen to twenty-four, and only 3.3 per cent had received the full two doses of MMR vaccine. Why did it affect these people? Because of a global vaccine shortage in the early 1990s. Mumps is not a harmless disease. I've no desire to scare anyone-and as I said, your beliefs and decisions about vaccines are your business; I'm only interested in how you came to be so incredibly misled-but before the introduction of MMR, mumps was the commonest cause of viral meningitis, and one of the leading causes of hearing loss in children. Lumbar puncture studies show that around half of all mumps infections involve the central nervous system. Mumps orchitis is common, exquisitely painful, and occurs in 20 per cent of adult men with mumps: around half will experience testicular atrophy, normally in only one t.e.s.t.i.c.l.e, but 15 to 30 per cent of patients with mumps orchitis will have it in both t.e.s.t.i.c.l.es, and of these, 13 per cent will have reduced fertility.

I'm not just spelling this out for the benefit of the lay reader: by the time of the outbreak in 2005, young doctors needed to be reminded of the symptoms and signs of mumps, because it had been such an uncommon disease during their training and clinical experience. People had forgotten what these diseases looked like, and in that regard vaccines are a victim of their own success-as we saw in our earlier quote from Scientific American Scientific American in 1888, five generations ago (see page 276). in 1888, five generations ago (see page 276).

Whenever we take a child to be vaccinated, we're aware that we are striking a balance between benefit and harm, as with any medical intervention. I don't think vaccination is all that important: even if mumps orchitis, infertility, deafness, death and the rest are no fun, the sky wouldn't fall in without MMR. But taken on their own, lots of other individual risk factors aren't very important either, and that's no reason to abandon all hope of trying to do something simple, sensible and proportionate about them, gradually increasing the health of the nation, along with all the other stuff you can do to the same end.

It's also a question of consistency. At the risk of initiating ma.s.s panic, I feel duty bound to point out that if MMR still scares you, then so should everything in medicine, and indeed many of the everyday lifestyle risk exposures you encounter: because there are a huge number of things which are far less well researched, with a far lower level of certainty about their safety. The question would still remain of why you were so focused on MMR. If you wanted to do something constructive about this problem, instead of running a single-issue campaign about MMR you might, perhaps, use your energies more usefully. You could start a campaign for constant automated vigilance of the entirety of the NHS health records dataset for any adverse outcomes a.s.sociated with any intervention, for example, and I'd be tempted to join you on the barricades.

But in many respects this isn't about risk management, or vigilance: it's about culture, human stories, and everyday human harms. Just as autism is a peculiarly fascinating condition to journalists, and indeed to all of us, vaccination is similarly inviting as a focus for our concerns: it's a universal programme, in conflict with modern ideas of 'individualised care'; it's bound up with government; it involves needles going into children; and it offers the opportunity to blame someone, or something, for a dreadful tragedy.

Just as the causes of these scares have been more emotional than anything else, so too has much of the harm. Parents of children with autism have been racked with guilt, doubt and endless self-recrimination over the thought that they themselves are responsible for inflicting harm upon their own child. This distress has been demonstrated in countless studies: but so close to the end, I don't want to introduce any more research papers.

There is one quote that I find-although she would perhaps complain about my using it-both moving and upsetting. It's from Karen Prosser, who featured with her autistic son Ryan in the Andrew Wakefield video news release from the Royal Free Hospital in 1998. 'Any mother who has a child wants it to be normal,' she says. 'To then find out your child might be genetically autistic is tragic. To find out that it was caused by a vaccine, that you agreed to have done...is just devastating.'

AND ANOTHER THING

I could go on. As I write this in May 2008, the media are still pus.h.i.+ng a celebrity-endorsed 'miracle cure' (and I quote) for dyslexia, invented by a millionaire paint entrepreneur, despite the abysmal evidence to support it, and despite customers being at risk of simply losing their money anyway, because the company seems to be going into administration; the newspapers are filled with an amazing story about a finger that 'grew back' through the use of special sciencey 'pixie dust' (I quote again), although the claim has been around for three years, unpublished in any academic journal, and severed fingertips grow back by themselves anyway; more 'hidden data' scandals are exposed from the vaults of big pharma every month; quacks and cranks continue to parade themselves on television quoting fantastical studies to universal approbation; and there will always be new scares, because they sell so very well, and they make journalists feel alive. could go on. As I write this in May 2008, the media are still pus.h.i.+ng a celebrity-endorsed 'miracle cure' (and I quote) for dyslexia, invented by a millionaire paint entrepreneur, despite the abysmal evidence to support it, and despite customers being at risk of simply losing their money anyway, because the company seems to be going into administration; the newspapers are filled with an amazing story about a finger that 'grew back' through the use of special sciencey 'pixie dust' (I quote again), although the claim has been around for three years, unpublished in any academic journal, and severed fingertips grow back by themselves anyway; more 'hidden data' scandals are exposed from the vaults of big pharma every month; quacks and cranks continue to parade themselves on television quoting fantastical studies to universal approbation; and there will always be new scares, because they sell so very well, and they make journalists feel alive.

To anyone who feels their ideas have been challenged by this book, or who has been made angry by it-to the people who feature in it, I suppose-I would say this: You win. You really do. I would hope there might be room for you to reconsider, to change your stance in the light of what might be new information (as I will happily do, if there is ever an opportunity to update this book). But you will not need to, because, as we both know, you collectively have almost full-spectrum dominance: your own slots in every newspaper and magazine in Britain, and front-page coverage for your scare stories. You affect outsider swagger, bizarrely, from the sofas of daytime television. Your ideas-bogus though they may be-have immense superficial plausibility, they can be expressed rapidly, they are endlessly repeated, and they are believed by enough people for you to make very comfortable livings, and to have enormous cultural influence. You win.

It's not the spectacular individual stories that are the problem, so much as the constant daily grind of stupid little ones. This will not end, and so I will now abuse my position by telling you, very briefly, exactly what I think is wrong, and some of what can be done to fix it.

The process of obtaining and interpreting evidence isn't taught in schools, nor are the basics of evidence-based medicine and epidemiology, yet these are obviously the scientific issues which are most on people's minds. This is not idle speculation. You will remember that this book began by noticing that there has never been an exhibit on evidence-based medicine in London's Science Museum.

A five-decade survey of post-war science coverage in the UK by the same inst.i.tution shows-and this is officially the last piece of data in the book-that in the 1950s science reporting was about engineering and inventions, but by the 1990 science reporting was about engineering and inventions, but by the 1990s everything had changed. Science coverage now tends to come from the world of medicine, and the stories are of what will kill you, or save you. Perhaps it is narcissism, or fear, but the science of health is important to people, and at the very time when we need it the most, our ability to think around the issue is being energetically distorted by the media, corporate lobbies and, frankly, cranks. everything had changed. Science coverage now tends to come from the world of medicine, and the stories are of what will kill you, or save you. Perhaps it is narcissism, or fear, but the science of health is important to people, and at the very time when we need it the most, our ability to think around the issue is being energetically distorted by the media, corporate lobbies and, frankly, cranks.

Without anybody noticing, bulls.h.i.+t has become an extremely important public health issue, and for reasons that go far beyond the obvious hysteria around immediate harms: the odd measles tragedy, or a homeopath's unnecessary malaria case. Doctors today are keen-as it said in our medical school notes-to work 'collaboratively with the patient towards an optimum health outcome'. They discuss evidence with their patients, so that they can make their own decisions about treatments.

I don't generally talk or write about being a doctor-it's mawkish and tedious, and I've no desire to preach from authority-but working in the NHS you meet patients from every conceivable walk of life, in huge numbers, discussing some of the most important issues in their lives. This has consistently taught me one thing: people aren't stupid. Anybody can understand anything, as long as it is clearly explained-but more than that, if they are sufficiently interested. What determines an audience's understanding is not so much scientific knowledge, but motivation: patients who are ill, with an important decision to make about treatment, can be very motivated indeed.

But journalists and miracle-cure merchants sabotage this process of shared decision-making, diligently, brick by brick, making lengthy and bogus criticisms of the process of systematic review (because they don't like the findings of just one), extrapolating from lab-dish data, misrepresenting the sense and value of trials, carefully and collectively undermining the nation's understanding of the very notion of what it means for there to be evidence for an activity. In this regard they are, to my mind, guilty of an unforgivable crime.

You'll notice, I hope, that I'm more interested in the cultural impact of nonsense-the medicalisation of everyday life, the undermining of sense-and in general I blame systems more than particular people. While I do go through the background of some individuals, this is largely to ill.u.s.trate the extent to which they have been misrepresented by the media, who are so desperate to present their favoured authority figures as somehow mainstream. I am not surprised that there are individual entrepreneurs, but I am unimpressed that the media carry their a.s.sertions as true. I am not surprised that there are people with odd ideas about medicine, or that they sell those ideas. But I am spectacularly, supremely, incandescently unimpressed when a university starts to offer BSc science courses in them. I do not blame individual journalists (for the most part), but I do blame whole systems of editors, and the people who buy newspapers with values they profess to despise. Specifically, I do not blame Andrew Wakefield for the MMR scare (although he's done things I hope I would not), and I find it-let's be very clear once again-spectacularly distasteful that the media are now revving up to hold him singly responsible for their own crimes, in regard to that debacle.

Similarly, while I could reel out a few stories of alternative therapists' customers who've died unnecessarily, it seems to me that people who choose to see alternative therapists (except for nutrition therapists, who have worked very very hard to confuse the public and to brand themselves as conventional evidence-based pract.i.tioners) make that choice with their eyes open, or at least only half closed. To me this is not a situation of businessmen exploiting the vulnerable, but is rather, as I seem to keep saying, a bit more complicated than that. We love this stuff, and we love it for some fascinating reasons, which we could ideally spend a lot more time thinking and talking about. hard to confuse the public and to brand themselves as conventional evidence-based pract.i.tioners) make that choice with their eyes open, or at least only half closed. To me this is not a situation of businessmen exploiting the vulnerable, but is rather, as I seem to keep saying, a bit more complicated than that. We love this stuff, and we love it for some fascinating reasons, which we could ideally spend a lot more time thinking and talking about.

Economists and doctors talk about 'opportunity costs', the things you could have done, but didn't, because you were distracted by doing something less useful. To my mind, the greatest harm posed by the avalanche of nonsense we have seen in this book is best conceived of as the opportunity cost of bulls.h.i.+t'.

We have somehow become collectively obsessed with these absurd, thinly evidenced individual tinkerings in diet, distracting us from simple healthy eating advice; but more than that, as we saw, distracting us from the other important lifestyle risk factors for ill health which cannot be sold, or commodified.

Doctors, similarly, have been captivated by the commercial success of alternative therapists. They could learn from the best of the research into the placebo effect, and the meaning response in healing, and apply that to everyday clinical practice, augmenting treatments which are in themselves also effective: but instead, there is a fas.h.i.+on among huge numbers of them to indulge childish fantasies about magic pills, ma.s.sages or needles. That is not forward-looking, or inclusive, and it does nothing about the untherapeutic nature of rushed consultations in decaying buildings. It also requires, frequently, that you lie to patients. 'The true cost of something,' as the Economist Economist says, 'is what you give up to get it.' says, 'is what you give up to get it.'

On a larger scale, many people are angry about the evils of the pharmaceutical industry, and nervous about the role of profit in healthcare; but these are formless and uncalibrated intuitions, so the valuable political energy that comes from this outrage is funnelled-wasted-through infantile issues like the miraculous properties of vitamin pills, or the evils of MMR. Just because big pharma can behave badly, that does not mean that sugar pills work better than placebo, nor does it mean that MMR causes autism. Whatever the wealthy pill peddlers try to tell you, with their brand-building conspiracy theories, big pharma isn't afraid afraid of the food supplement pill industry, it of the food supplement pill industry, it is is the food supplement pill industry. Similarly, big pharma isn't frightened for its profits because popular opinion turned against MMR: if they have any sense, these companies are relieved that the public is obsessed with MMR, and is thus distracted from the other far more complex and real issues connected with the pharmaceutical business and its inadequate regulation. the food supplement pill industry. Similarly, big pharma isn't frightened for its profits because popular opinion turned against MMR: if they have any sense, these companies are relieved that the public is obsessed with MMR, and is thus distracted from the other far more complex and real issues connected with the pharmaceutical business and its inadequate regulation.

To engage meaningfully in a political process of managing the evils of big pharma, we need to understand a little about the business of evidence: only then can we understand why transparency is so important in pharmaceutical research, for example, or the details of how it can be made to work, or concoct new and imaginative solutions.

But the greatest opportunity cost comes, of course, in the media, which has failed science so spectacularly, getting stuff wrong, and dumbing down. No amount of training will ever improve the wildly inaccurate stories, because newspapers already have specialist health and science correspondents who understand science. Editors will always-cynically-sideline those people, and give stupid stories to generalists, for the simple reason that they want stupid stories. Science is beyond their intellectual horizon, so they a.s.sume you can just make it up anyway. In an era when mainstream media is in fear for its life, their claims to act as effective gatekeepers to information are somewhat undermined by the content of pretty much every column or blog entry I've ever written.

To academics, and scientists of all shades, I would say this: you cannot ever possibly prevent newspapers from printing nonsense, but you can add your own sense into the mix. Email the features desk, ring the health desk (you can find the switchboard number on the letters page of any newspaper), and offer them a piece on something interesting from your field. They'll turn you down. Try again. You can also toe the line by not writing stupid press releases (there are extensive guidelines for communicating with the media online), by being clear about what's speculation in your discussions, by presenting risk data as 'natural frequencies', and so on. If you feel your work-or even your field-has been misrepresented, then complain: write to the editor, the journalist, the letters page, the readers' editor, the PCC; put out a press release explaining why the story was stupid, get your press office to harra.s.s the paper or TV station, use your t.i.tle (it's embarra.s.sing how easy they are to impress), and offer to write them something yourself.

The greatest problem of all is dumbing down. Everything in the media is robbed of any scientific meat, in a desperate bid to seduce an imaginary ma.s.s who aren't interested. And why should they be? Meanwhile the nerds, the people who studied biochemistry but who now work in middle management at Woolworths, are neglected, unstimulated, abandoned. There are intelligent people out there who want to be pushed, to keep their knowledge and pa.s.sion for science alive, and neglecting them comes at a serious cost to society. Inst.i.tutions have failed in this regard. The indulgent and well-financed 'public engagement with science' community has been worse than useless, because it too is obsessed with taking the message to everyone, rarely offering stimulating content to the people who are already interested.

Now you don't need these people. Start a blog. Not everyone will care, but some will, and they will find your work. Unmediated access to niche expertise is the future, and you know, science isn't hard-academics around the world explain hugely complicated ideas to ignorant eighteen-year-olds every September-it just requires motivation. I give you the CERN podcast, the Science in the City mp3 lecture series, blogs from profs, open access academic journal articles from PLOS, online video archives of popular lectures, the free editions of the Royal Statistical Society's magazine Significance Significance, and many more, all out there, waiting for you to join them. There's no money in it, but you knew that when you started on this path. You will do it because you know that knowledge is beautiful, and because if only a hundred people share your pa.s.sion, that is enough.