Sounding the alarm on publication bias
by 
File photo courtesy: Unsplash
Achieving success after a series of failures isn’t an anomaly: Past research has shown it’s a healthy part of the process. Understanding (and embracing) this concept can make for more focused scientists and help combat impostor syndrome, an affliction that could affect as many as 20 per cent of post-secondary students.
But in several STEM fields, academic papers are overwhelmingly positive, creating the false impression that experiments always go as planned. This practice is called publication bias, and critics say it fuels insecurity and limits opportunities for learning and development.
In a recent article in Nature’s career column, Devang Mehta, a postdoctoral researcher in plant genomics at the University of Alberta, argues in favour of publishing negative results, highlighting an April paper he and his colleagues wrote.
RELATED: The link between failure and long-term success
The study, titled “Linking CRISPR-Cas9 interference in cassava to the evolution of editing-resistant geminiviruses,” represents a failed experiment in trying to use the CRISPR gene-editing tool to make cassava crops resistant to a damaging viral disease.
“Despite previous reports that CRISPR could provide viral immunity to plants by disrupting viral DNA, our experiments consistently showed the opposite result,” Mehta writes.
“Our paper also showed that using CRISPR as an ‘immune system’ in plants probably led to the evolution of viruses that were more resistant to CRISPR. And although this result was scientifically interesting, it wasn’t the ‘positive’ result that applied scientists like me are taught to value.”
While the peer reviewers agreed the study was methodologically sound, Mehta says there was resistance toward publishing it.
“Scientists have become so accustomed to celebrating only success that we’ve forgotten that most technological advances stem from failure,” he writes.
“Even if my research was flawed, the problem remains that the scientific world largely ignores negative results.”
The negative impact of publication bias
Mehta isn’t the first to sound the alarm on publication bias.
A 2012 study suggests the frequency in which positive-results papers are published is increasing, and that could “discourage high-risk projects and pressure scientists to fabricate and falsify their data.”
A 2018 study on dental papers found similar results.
“Dental researchers believe that the negative data have no value and all the journals maintain that attitude,” reads an excerpt from the paper.
“They are also well aware of the fact that editors exhibit less interest in publishing studies that do not depict the previously available data.”
There are a few reasons publication bias exists — funding agencies that prefer positive results and journals’ hesitation to publish failures being two of them.
But that may be doing more damage than good and can hinder long-term progress.
“Publication bias has an escalating and damaging effect on the integrity of knowledge,” Dr. Ridha Joober, MD, Ph.D. et al wrote in a 2012 paper.
In the case of clinical trials, “withholding negative results from publication … could have major consequences for the health of millions. In preclinical and experimental research, this bias may seriously distort the literature, drain scarce resources by undertaking research in futile quests and lead to misguided research and teaching practices … research participants consent to participate in research on the understanding that they are contributing to advances in treatment and scientific knowledge.”
In his Nature article, Mehta calls on reviewers and publishers to put honesty first.
“We need academic conferences to embrace honest discussions of failed experiments. We need funding agencies to support scientists who produce sound negative results,” he says.
“And, as scientists, we must acknowledge that all important work should be recognized, irrespective of its outcome.”
