Best practices in replication: A case study of common information in coordination games

Presenter

Yan Chen

School of Information, University of Michigan

Authors: Roy Chen, Yan Chen, Yohanes E. Riyanto

Time and location

North Quad 4330, Thursday (1:00-2:00) pm

Abstract

Recently, social science research replicability has received close examination, with discussions revolving around the degree of success in replicating experimental results. We lend insight to the replication discussion by examining the quality of replication studies. We examine how even a seemingly minor protocol deviation in the experimental process (Camerer et al. 2016), the removal of common information, can lead to a finding of “non-replication” of the results from the original study (Chen and Chen 2011). Our meta-analysis of the data from the original study, its replication, and a series of new experiments shows that, with common information, we obtain the original result in (Chen and Chen 2011), whereas without common information, we obtain the null result in Camerer et al. (2016). Together, we use our findings to propose a set of procedure recommendations to increase the quality of replications of laboratory experiments in the social sciences.