Speaker: John P.A. Ioannidis
Subject: How to improve the scientific process.
Location: Erasmus University College
Author: Kristian Blom
Today 09-11-2015 is a rather special day for the Erasmus University, since they celebrate their 102nd dies natalis. For this special event, Prof. John P.A. Ioannidis, Chair in Disease Prevention and Professor of Medicine at Stanford University, gave a masterclass about the efficiency of published research. Later that day he received an honorary doctorate of the university.
Prof. Ioannidis started the masterclass by mentioning that in the period of 1996 to 2011 more than 15,153,000 different scientists have published their research in major scientific journals. But of all these published researches, just a few several thousand were major discoveries. Another thing that is stated in one of his articles is that it can be proven that most claimed research findings are false. So what we see is that the efficiency of research findings is rather low. One way to define the efficiency is the ratio between correctness (quality) and the time it took to do the research (quantity).
If we want to increase the efficiency in published research, we need to know which measures leads to the maximum efficiency.
For instance, if we make reanalysis mandatory, what would happen with the efficiency? One can say that the efficiency shall increase, because reanalyzing your data will give less errors in published research. On the other hand, more time will be spent on the same research, thereby the efficiency will decrease. So by increasing the quality of research, we automatically decreasing the quantity.
If we look back at the number of scientist who have published their research between 1996 and 2011, we can say that a decrease in quantity isn’t that worse, since the equilibrium is more shifted to the quantitative side than the qualitative side. So in general, making reanalysis mandatory should increase the efficiency of published research.
But when reanalysis is done, how do you know whether the new/old claimed research finding is true or not? This question is very difficult to answer. Sometimes it can occur that a claimed research finding is accepted as true, but decades later people find out that the finding actually was false.
For example: About two weeks ago Prof. Ronald Hanson published an article in Nature were he showed that quantum entangled electrons in two diamonds can send information to each other instantaneously. This founding showed that Einstein was wrong about his idea that quantum entanglement isn’t possible. Although it was already accepted that quantum entanglement exists, it took 80 years of research before researchers were able to prove it.
One thing that Prof. Ioannidis found out was that the efficiency of research increases with the amount of pre-study that is done (see image 1). Of course this makes sense, but what is less obvious is that the PPV (probability that a research finding is true) doesn’t depend on the power (the importance of a study in the scientific world). So that indicates that no matter how important/famous a research is, it is always important as a researcher to do enough pre-study before you start with the real work.
‘Where will science be in 10 years?’ was one of the last questions during the masterclass. Prof. Ioannidis gave as answer: ‘There will be a great increase in researchers, and hopefully many great discoveries will be achieved.’ Of course this is something we all hope, but therefore alterations in the scientific process are necessary for improvement. Not only for an increase in efficiency as we discussed here, but also for keeping science as successful as it is nowadays.
Article: Why most published research findings are false.
Author: John P.A. Ioannidis