Reproducibility in Scientific Research
Monya Baker
May 4, 2021 at 2:00 P.M. (video)
Abstract
Everyone agrees that reproducibility in scientific research is a crucial issue in need of improvement. Few agree on what reproducibility actually means.
Over 15 years ago, an epidemiologist proposed that publication requirements favoring surprising results combined with naïve statistical thresholds meant that most publications were false. Since that conclusion relied on modelling, it was easy to dismiss. In 2011 and 2012, researchers at Bayer and Amgen described their efforts to reproduce dozens of breakthrough papers in cancer, women’s health and heart disease: they could verify fewer than a fourth of the topline conclusions. A more systematic effort to replicate psychology studies succeeded with only about a third replicated. That spurred what some have called a ‘credibility revolution:’ changes by funders, journals, and individual scientists to make studies more reliable. These include better reporting of methods, data sharing, disclosure of negative results, and changes in peer review practices and incentives.
However, progress bogs down because the meaning of reproducibility varies across disciplines and from scientist to scientist. More prosaically science is hard; egos get in the way. And science proceeds more from hint to hint than from pronouncement to pronouncement. Striving for 100% reproducibility could stall the dissemination of cutting-edge but less vetted ideas. This talk explores the advances and challenges in improving scientific reproducibility.
Speaker
Monya Baker commissions and edits articles on improving science for Nature magazine, where she has worked since 2007. Her work has appeared in Nature, Science, Wired, The Economist, Slate, New Scientist and elsewhere. She has an Ed.M from Harvard University and a B.A. in biology from Carleton College.