Many Labs 5: Can conducting formal peer review in advance improve reproducibility?
Projektlaufzeit: 2016-2020
Projektleitung: Dr. Charles Ebersole
Kooperationspartner: Dr. Charles Ebersole, University of Virginia; Prof. Joshua Hartshorne, Boston College; Dr. Thomas Schultze & Prof. Dr. Stefan Schulz-Hardt, Universität Göttingen, …
Replication efforts in psychological science sometimes fail to replicate prior findings. If replications use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the replication protocol rather than a challenge to the original finding. Formal pre-data collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replications from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) in which the original authors had expressed concerns about the replication designs before data collection and only one of which was “statistically significant” (p < .05). Commenters on RP:P suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these failed to replicate (Gilbert et al., 2016). We revised the replication protocols and received formal peer review prior to conducting new replications. We administered the RP:P and Revised replication protocols in multiple laboratories for high-powered tests of each original finding with both protocols.
Literatur
Ebersole, C. R., Mathur, M. B., …, Rentzsch, K., …, & Nosek, B. A. (2020). Many Labs 5: Testing pre-data collection peer review as an intervention to increase replicability. Advances in Methods and Practices in Psychological Science, 3(3), 309-331 https://doi.org/10.1177/2515245920958687
Skorb, L., Aczel, B., …, Rentzsch, K., …, & Hartshorne, J. K. (2020). Many Labs 5: Replication of van Dijk, van Kleef, Steinel, and van Beest (2008). Advances in Methods and Practices in Psychological Science, 3(3), 418-428. https://doi.org/10.1177/2515245920927643
Kontakt: Katrin Rentzsch | k.rentzsch@phb.de | Tel.: 0176-34465882