Violent metaphors

 

I haven’t written here about the CDC “whistleblower” issue, because I was in Shanghai when the story broke with both limited internet access and limited desire to take time away from adventures to write. Orac did an excellent job of staying on top of the story, and I refer the interested reader to his series of posts on the subject, as well as this excellent summary by Todd W. at Harpocrates Speaks, and this one by Retraction Watch.

However, as many people who read Violent Metaphors have a specific interest in vaccine/anti-vaccine issues, I thought it would be worth talking about the most recent development in the story; specifically, the retraction of Brian Hooker’s journal article purporting to show an increased risk of autism among African American boys who receive the MMR vaccine.

For the last few months, the antivaccine community has been touting this study, which reanalyzes a dataset used by DeStefano et al. in their 2004 publication “Age at First Measles-Mumps-Rubella Vaccination in Children With Autism and School-Matched Control Subjects: A Population-Based Study in Metropolitan Atlanta”. In the DeStefano study (which is unfortunately paywalled), the authors conducted a case-control study compared MMR vaccination histories of children with (cases) and without (controls) autism. DeStefano et al. reported that

the overall distribution of ages at first MMR vaccination among children with autism was similar to that of school-matched control children who did not have autism. Our hypothesis was that earlier age at vaccination, ie, before a possible critical time window for autism development, might be associated with an increased risk for autism. When we analyzed associations according to different age cutoffs, we found that similar proportions of case and control children had been vaccinated before 18 months or before 24 months of age. No specific associations for either of these age cutoffs were found for specific subgroups of case children, including children with some indication of regression or plateau in development, the group of most concern based on the clinical reports of Wakefield et al. (1998). (pp262-263)

After being contacted by William Thompson (a CDC researcher and the third author of the DeStefano study)—more on this below–Hooker reanalyzed the dataset used by DeStefano et al., which is curated by the CDC. However, he analyzed the data in a different way than DeStefano et al. Instead of using a regression analysis, he sliced the dataset into many tiny groups and performed a Chi-square analysis to look for statistically significant associations between timing of vaccination and autism. The only group he found to have a significant association was African American boys between the age of 24 and 35 months, who had a 3.4 fold increased risk of autism.

“The results show a strong relationship between child age at the administration of the first MMR and autism incidence exclusively for African American boys which could indicate a role of the vaccine in the etiology of autism within this population group. This particular analysis was not completed in the original Destefano et al. [14] (CDC) study. Although the previous study considered MMR timing and African Americans in general, no statistically significant effect was observed. This is in contrast to our result for African Americans in general, because the CDC study limited the total African American cohort to include only those individuals who possessed a valid State of Georgia birth certificate which decreased the statistical power of their analysis.”

But Hooker’s reanalysis was incompetent and misleading for several reasons. First, the DeStefano paper didn’t leave out African American children in order to hide this association, as he alleges. Their exclusion of children from the analysis followed an overall experimental protocol, and didn’t target African American children specifically. As Reuben Gaines at The Poxes Blog explains:

Children were excluded from the analysis because of very legitimate and scientific reasons. They either were not the right age, did not have autism but some other neurodevelopment disorder, or were born outside of Georgia. Even if they were tossed into the analysis, DeStefano et al used a statistical analysis that took into account things like birth weight and mother’s age when analysing the data. They wanted to make sure that what they were seeing was most likely because of the MMR vaccine and not because of some other factor associated with autism.

Secondly, instead of doing a case-control study (which is what the dataset was designed for), he did a cohort study and sliced the dataset into tiny, tiny sub-groups. If you make a sample size small enough, you can turn up all kinds of interesting “significant” associations, which mean nothing biologically. It’s a bit like taking a jar of a million red and blue marbles and drawing a thousand of them to see if the distribution is random: you can look at the entire group for significant patterns, or split it up into tiny group of five and pretend it’s a big deal when you find a group that’s all red. (Edited to add: In fact, Hooker didn’t even publish the sample sizes of the groups he was analyzing. They could have been 100 children, they could have been 6 children….we simply don’t know. I agree with @david_colquhoun that this omission is “disgraceful”).

Finally, the Chi-square test he used was a completely inappropriate analytical method, as it can’t account for confounders (like birth weight) that could potentially skew the results. Only a regression analysis, such as the one DeStefano et al. originally performed, allows the effects of these potential confounders to be studied in the complete sample.

The publication of Hooker’s paper in Translational Neurodegeneration was touted by a few anti-vaxxers as evidence of a CDC cover up of the association between MMR and autism (you can find many of their comments by searching the twitter hastags #HearThisWell and #CDCWhistleblower). Andrew Wakefield capitalized on the situation by posting an outrageous, race-bating video (comparing Hooker’s findings to the Tuskeegee syphilis experiment)**.

In his desperate efforts to stay relevant by inserting himself into a new anti-vaccine narrative, Wakefield helped label Thompson (the CDC researcher who was an author on the DeStefano et al. 2004 study) as a “whistleblower” against Thompson’s own wishes. In a statement released through his lawyer, Thompson said

“I have had many discussions with Dr. Brian Hooker over the last 10 months regarding studies the CDC has carried out regarding vaccines and neurodevelopmental outcomes including autism spectrum disorders. I share his belief that CDC decision-making and analyses should be transparent. I was not, however, aware that he was recording any of our conversations, nor was I given any choice regarding whether my name would be made public or my voice would be put on the Internet.”

The back-and-forth about the “CDC whistleblower” is complex, and played out largely on social media. If you would like to delve into it, here’s an excellent analysis and timeline. I won’t attempt to go into all the details here, as they would make for quite a long post, and a great deal of it is unconfirmed rumor and speculation. However I will note that the only thing that Thompson has said publicly on the subject was the statement through his lawyer, and included the following sentences, frantically ignored by antivaxxers on twitter and elsewhere:

“I want to be absolutely clear that I believe vaccines have saved and continue to save countless lives. I would never suggest that any parent avoid vaccinating children of any race. Vaccines prevent serious diseases, and the risks associated with their administration are vastly outweighed by their individual and societal benefits.”

 

I want to instead focus on the fact that Hooker’s study has now been retracted by Translational Neurodegeneration. In their statement, the journal editors noted that it was retracted for two reasons: 1) “undeclared competing interests on the part of the author which compromised the peer review process.”, and 2) “post-publication peer review raised concerns about the validity of the methods and statistical analysis”. We’ve already discussed the second point, but what does the first mean?

BioMed Central, the publisher of Translational Neurodegeneration has, like all other respectable journal publishers, a requirement that authors declare their competing interests upon publication. C.O.P.E. (the Committee On Publication Ethics, of which BioMed Central is a member) explain competing (also known as conflicting) interests as:

Conflicts of interest arise when authors, reviewers, or editors have interests that are not fully apparent and that may influence their judgments on what is published. They have been described as those which, when revealed later, would make a reasonable reader feel misled or deceived. (Guidelines on Good Publication Practice (2003) )

These conflicts are often financial. If someone has the potential to make (or have made) a financial (or personal) gain if the outcome of their research goes a certain way, they may (even unknowingly) be influenced to skew their results that direction. Examples of conflicts may be receiving funding from a company, or accepting speaker’s fees.

For this reason, journals require authors to openly declare any potential competing/conflicting interests upon the submission of a manuscript. Importantly, the presence of a competing interest doesn’t necessarily guarantee that the paper will be rejected–it’s up to the judgment of the editors—but failure to disclose such an interest is a huge red flag of potential misconduct on the part of the author(s). It’s a very serious thing indeed.

It’s not clear at this point what conflicts of interest Hooker has failed to disclose, but Harpocrates Speaks notes that:

Hooker has an open case claiming vaccine injury for his son before the Vaccine Court. He is also a board member of an anti-vaccine organization called Focus Autism. Focus Autism is the organization that funded Hooker’s study. Hooker is also ideologically opposed to vaccines and the CDC, as evidenced by an email he sent to former director of the CDC, Julie Gerberding, in which he wrote, “I would personally urge you to review the Book of Matthew 18:6 and consider your own responsibility to all children of the U.S. including my own son.” The referenced Bible passage reads (King James Version):

“But whoso shall offend one of these little ones which believe in me, it were better for him that a millstone were hanged about his neck, and that he were drowned in the depth of the sea. “

We will have to await further details, and I expect we’ll see them first on Retraction Watch which has contacted BioMed Central for a comment.

In the meantime, be aware that there is a significant chance that the antivaccine community will try to spin this retraction as part of the alleged CDC/ vaccine manufacturer conspiracy. It’s not. Peer review is essential to the scientific process, but it is not infallible. Reviewers make mistakes, and things get published that shouldn’t. But over time, thanks to efforts to replicate scientific studies and to post-publication peer review, these errors tend to become evident, with the result that papers get corrected or retracted. (I highly recommend that everyone check out Retraction Watch for valuable insights into this process).

This retraction is not conspiracy, but rather evidence of the process working as it should. Wakefield’s paper was retracted because of fraud and ethical lapses, and Hooker’s paper has been retracted because of undisclosed competing interests and shoddy, shoddy methods. While the antivaccine movement is trying to cloak itself in science, it persists in refusing to hold itself to the same scientific standards that everyone else is held to. Allegations of conspiracy are simply feeble attempts at misdirection, to hide the fact that their “research” is incompetent, unethical, and fraudulent.
——————————————————————————————————————————–

**Orac points out that Wakefield’s touting of Hooker’s study is strange because it completely contradicts Wakefield’s own research.

Notice how he completely neglects to mention that in every other subgroup [examined in the study], even Hooker couldn’t torture the data to make it confess a relationship between age at MMR vaccination and autism in any other population other than a very small population in the study: African-American males. Whenever that happens as you slice epidemiological data finer and finer, you should be alert for the very distinct possibility that what you’re really looking at is a spurious correlation. As I pointed out before, Hooker in reality merely confirmed that Wakefield was wrong about everyone except African-American males, and, given how small this subgroup was in the study, almost certainly didn’t find any evidence supporting Wakefield’s hypothesis (such as it is) for even African-American boys. Yet, Wakefield, as deluded as he is, spins it as “vindication.”