The Lancet has formally retracted the 1998 paper by Wakefield which originally asserted a link between vaccines and autism. This is fantastic news and a vindication of the scientific method and peer review.
Lots more details at Ars Technica about this.
{ 24 comments }
“This is fantastic news and a vindication of the scientific method and peer review.”
No it isn’t.
While I agree that the correlation between vaccines and and autism wasn’t even tenuous and that it was no where near being proved; I do not think this is a “vindication” of the peer review process.
Unlike AGW, the vaccine-autism link was always promoted by non-scientists (and a few scientists) on the fringe; they weren’t the status quo in belief. So it wasn’t really a vindication since people in the peer review process never bought the link; unlike how they bought AGW hook-line-and-sinker.
Is it really a vindication since it took 12 years to retract the paper?
TexasAgo3
let me defend the peer review process by saying that good science does take time to insure that no biases are entering the decision making process.
But I still think that if the situation had been flipped; that the people who didn’t believe in the link were in the fringe and the proponents of the theory were in the mainstream, the status quo would be to not under any circumstances vaccinate your child.
Mike,
I agree and understand that it takes time (I’m an engineer myself). I just wonder if 12 years is reasonable in this particular case. I honestly don’t know – I am just curious.
I’m failing to see how this vindicates peer review. Isn’t The Lancet peer reviewed? And didn’t peer review therefore fail to correct this paper?
Illuminate my ignorance, someone. It is even more vast than the bloated ball of opinions that roils my ego.
Yours,
Tom
I think this has little to do with peer review. Peer review is not intended to catch outright fraud. In this case one of the authors lied about how he gathered the data in his paper.
What happened is other scientists tried to reproduce the results described in this paper and were not able to reproduce them. That is the core of the scientific method. You do an experiment or study, publish your method and the results, then other scientists try to repeat your experiment or study to see if they get similar results.
When no one could reproduce the results, people started to question the methods that were used and eventually uncovered the fraud.
Mikeca — an overstatement, perhaps? If not, then what on earth good is peer review?
Its a vindication because it *did* catch outright fraud. The work was published, and then the sscientific establishment tried to reproduce the results and failed. In the meantime much evidence accumulated that contradicted the study. Other scientists published and did research and in the end, the Lancet study was deprecated. The Lancet retraction is just the final formal straw but the overall process of science testing the fraudulent conclusion worked and worked really well.
yes it took over a decade. Thats because science doesnt spin on a budget cycle and research takes time. I think that in this case it was actually pretty speedy.
Okay,
I surrender to the validity qua science as such.
So how many years will it take science to nullify Al Gore’s :
An Inconvenient Truth: The Planetary Emergency of Global Warming and What We Can Do About It
That’s a question just in case there might be a question.
Peer review isn’t designed to catch outright fraud; it can but it can’t be expected to. There are, in my opinion, severe flaws in the current model of peer review, most especially in any area where there is serious money involved. But saying it failed to catch an outright fraud isn’t entirely fair, since it can’t be expected to deal well with someone just flat out making crap up. In fact, arguably there’s a hole in it, because if someone DOES flat out make crap up, but enough people believe the crap, a whole lot of useless cycles can be wasted trying to prove the crap isn’t crap even though it is. Indeed, you can even see a whole weighty bunch of useless research based on a bad premise.
Peer review is supposed to be where you can publish a study, have it looked at by your peers to make sure you haven’t goy any obvious problems before publication, and then, open it to criticism to other peers once it’s published. But if I publish a study on werewolves using werewolf-evidence that my peers have to admit looks pretty good, and I’m the only one who knows I faked my data, then, it’s likely going to get through.
The scientific method isn’t perfect; but when it works properly, it’s self correcting. This is indeed a vindication of the scientific method. Peer review? Not so much, but that’s only one aspect of the larger process of science. Science corrected itself here.
I wonder if anyone has told Jenny McCarthy yet? Somehow I doubt she’s a Lancet subscriber.
I’m with Martin…
It’s the scientific method that worked, not peer review.
Peer review is useful, but not an end-all-be-all.
ive never claimed peer review is the end-all. Its like democracy – it sucks pretty much all the time, actually. But everything else sucks worse.
the “flaws” in peer review are such tha if you tried to solve them 0- for example, instead of letting only people in the subfield be reviewers of relevant papers and instead recruited outsiders – science would slow to a crawl and the quality of papers would be much worse. I can definitely snow a reviewer of my work if I know they are therapy physcs guys and not imaging physics guys. I can even snow most imaging physics guys who do CT as opposed to MRI like I do. And Im pretty sure it would be harder but not impossible to snow a guy doing fMRI or angio MRI, if my paper is about DTI MRI or fat-water MRI. But if my paper is on fat-water MRI, and a fat-water MRI guy is revewing it (double blind, of course), then i am not gonna get away with much.
Wakefields study, as Ars points out, had no actual science in it. The link i gave in my post is really interesting and gives a lot of detail and insight into just how Wakefield pulled it off.
McK, i think you raise an interesting point (though you will disagree). On every issue there is indeed a scientific consensus, based on scientific work, which is performing using the scientific method and judged by peer review. That consensus can change but it is a slow process as old theories are gradually refined and changed – usually as a result of better experiments, not because the old experiments which supported the old status quo were flawed, but more because our understanding of how to test the hypothesis has changed.
In all cases the quality of peer review may vary (are papers judged dbl-blind? etc). But generally the consensus is is rooted in solid ground and the part of it that we the lay public can see is like na iceberg – 99% invisible to our eyes, 1% above the waterline. Its easy for us to critique that 1% – to paint the iceberg blue – but the peole in the field wo work with teh cinsensus see the whole thing.
That doesnt mean that the skeptics on a given issue are wrong a-priori. It means though that hanging teh consensus requires more work than any outsider can do.
The reason it’s not a vindication is that no one has said that peer review never works.
Therefor a case of it working doesn’t refute anything.
The argument is that peer review can (but again, not always) fall prey to conflict of interest problems. And that because funding can (again, not always) become circular the problem can become institutionalized.
No, this is misplaced scientific “triumphalism”.
It would be fantastic news if scientists found the cure for autism.
It would be important news if scientists found the cause of autism.
Haggling over Wakefield’s paper is pure scientific politics.
I say this not as anti-vaxxer. I have no problems with vaccines. In theory, they are wonderful. In practice, they probably confer more benefit, than harm.
But, the primary issue is, What is causing all this autism?
–HB
“I can definitely snow a reviewer of my work if I know they are therapy physcs guys and not imaging physics guys”
That’s an interesting point. Perhaps peer review should be expanded to include people who aren’t experts on the subject, but are on method. A lot of the weaknesses that have killed AGW claims are from people who don’t know squat about climate science, but who know programming and statistical analysis and are qualified to say the statistical data couldn’t show what the researchers claim it does, even if the research was valid. I suspect this problem is more common than scientists like to admit.
Hank: I don’t think “probably” is justified; I think “beyond any plausible doubt” is.
As to what’s causing “all this autism”, the best hypothesis I’m aware of is that the rate of autism isn’t increasing (and has not increased) at all.
What has increased is diagnosis of it, due to a combination of looking for it more strenuously and the “spectrum” that counts as “autism” being spread wider over the years.
My understanding is that today “autism spectrum disorders” cover things that nobody would have called “autism” a few decades ago.
(Desire for a diagnosis for special benefits is probably a part of that, but I reckon a small one.
Mostly I imagine it’s the psych doctors looking at cases that are plausible edge cases in the current diagnosis and deciding [plausibly] that the should count.
As that goes on, you get a broader definition.)
(And when I say “the rate of autism isn’t really increasing”, I mean “the rate according to the diagnostic system in use at the starting time T”; the “rate” is increasing, of course, but due to changes in the diagnostic system, not changes in the underlying data.)
And if the rate of autism is really increasing, it is potentially due in part to the increased reproductive success of nerds over the last few decades.
I know that in finance and business it is difficult for even good auditors to find fraud, especially if you get other key actors to conspire. It’s not surprising that it is equally difficult in other fields.
These are just very complex, and when you get down to it, very narrowly specialized situations, that only a few people (at a given moment) have a complete understanding of.
The truth comes to light eventually, but it can take time.
And peer review isn’t even an audit in the business sense. Auditors take their own notes, gather their own data, and then apply their own methods to that data and also to the data gathered by the business. They observe and research. Peer reviewers review (obviously) the data and methods and standards applied.
I know grant organizations commonly have auditors; but what I don’t know is if they audit just the finances, or also the methods and data. Aziz, can you enlighten me? It seems like an auditing system would’ve been helpful in this case; but I have to believe that widespread auditing would be expensive and difficult, and would slow research in general. (Of course, slowing BAD research would be a good thing.)
J1, its a misconception to thinkthat only statistics guys can do stats. The stats guys all specialize and often the experts for a given subfield are the nonstas guys who have had to learne and develop their own stats tailored to the application. Just on eexample I can give you is how we measure noise in MRI – there’s all sorts of things that we have to do – driven by the physics of the MR signal – that a stats guy would be utterly flabberasted by. I know. I had one on my advisory committee to fill a “slot” the grad school insisted I fill. He was dead weight. (for specifics, if you have notingbetter to do and are a masocist, check out teh paper by Gudbjartsson et al about “Rician noise distribution in MRI”. Its on pubmed.com, the abstract is free).
Having interdiscinplinary breadth is a good thing when your ambition is genuinely interdisciplinary. Example would be a biotech reseach facility – youd want people running it from across the spectrum of fields. But for a aper in a subfield, the best people to do the analysis arethe experts themselves.
Yu may be interested to know that almost all of the major advances in statistics – from the t-test on up – were almost all developed by non-stats people, but rather physics or other science people. The t-test itself was developed by a chemist, in fact – and he wasnt even doing chemistry but was vetting applicants for a job! at least that is the lore I’veheard.
Martin,
im writing a grant right now and Ican tell you that they want to know everything. They want to know exacly (down to teh dollar) how we are spending the money. They want to know what money we already have. They want to know the physics and the science, they want to know the IP issues and the licensing, they want to know the market and the public domain applicability. Its like being audited and being peeer reviewed at the same time, except of ocurse for agrant you are asking for money TO do research, not being vetted on research DONE (though you usually have prelim data which is indeed “reviewed” for itsapplicability and relevance).
Its a nightmare and rightly so. The stimulus package Obama signed sent money to all corners of science, and in every case the number of applicants for the handful of grants that were available were 10-1 or worse. VEryone and theu rmother leatpt to apply and the applicant succes rate was far, far below what is normal. Thats going to be the cream of teh crop.
Comments on this entry are closed.