Three Obstacles that Hinder Reproducibility in Research

laboratory rat looking scared trapped in a cage

Scientific advancement takes place when experiments produce results that are reliable and reproducible, so that data can be used confidently by the scientific community. But there is compelling evidence that a reproducibility crisis continues to plague basic and preclinical research, as it is estimated that only 11-25% of experiments are able to be successfully replicated.

Stanford Professor John Ioannidis addressed this issue in a recent Journal of the American Medical Association article. Ioannidis identified three main issues that contribute to nonreproducibility in preclinical research, including experiments performed on animals. He also offered potential fixes for these problems:

  1. Misaligned reward/incentives system: With the current “publish or perish” mentality, scientists are motivated to publish their research findings quickly to build up their resumes, and are rewarded for having a large publication record. What if, Ioannidis suggests, potential funding agencies or those in positions to hire or promote scientists instead focused on whether researchers have a good record of sharing data and experimental protocols? What if scientists were rewarded for taking steps to ensure their research was conducted to the highest standards? Putting less focus on the number of publications and more emphasis on transparency and quality could help reproducibility.
  2. Poor research methods: The manner in which animal experiments are conducted and reported can lead to reproducibility issues—topics that we’ve discussed many times in Science First. For example, many animal studies lack bias-reducing measures, do not include proper statistical tests, lack information on why animals are dropped from studies, and do not note whether or not pain relievers were used. Improving the design, execution and reporting of animal experiments will improve reproducibility.
  3. Lack of transparency: Scientific publications often omit important details regarding the methodology researchers use in their experiments. This omission potentially allows scientists to modify methods after they begin a study in order to get desired outcomes. Encouraging researchers to register their studies ahead of performing their research would help address this, as it would require important details like sample size, experimental controls and planned statistical analyses to be documented before experiments begin. As a result, researchers would likely design and report on their studies more carefully, knowing that others in the scientific community could compare details from the registry to what is actually published.

Considering these issues have resulted in a widespread reproducibility crisis and yet still remain common in the scientific community, Ioannidis stated, “The research community should reassess whether it can have the luxury of continuing to fund so much research that is nonreproducible.”

We propose that a different reassessment be made: whether the research community should have the luxury of continuing to work with animal models at all. Issues with animal models extend beyond study design and transparency and include inherent problems, such as extrapolating data across species. If the scientific community really wants to improve human health and well-being through research, then efforts need to be refocused on human-relevant models that have a higher probability of improving human health.

Source: Ioannidis, J. “Acknowledging and Overcoming Nonreproducibility in Basic and Preclinical Research” JAMA, Feb. 13, 2017.

This entry was posted in News and tagged on March 13, 2017.
Comments are closed.