Spurious Precision in Meta-Analysis of Social Science Research (METASPUR)

I argue that the meta-analysis methodology used in observational research entails a bias, one that stems from a reliance on reported precision which often exaggerates real precision. I will show that, when applied to all meta-analysis contexts, a method inspired by instrumental variables used in economics corrects for the bias. In meta-analysis, studies reporting more precision get more weight. Yet more than 20,000 metaanalyses have been conducted in social sciences, mostly synthesizing observational research, where researchers have control over the precision measure they report. Hence, reported precision is endogenous.
I challenge the assumption that observational studies reporting more precision are more informative and less biased. While I focus on social sciences, the results will apply to the 100,000+ meta-analyses in other fields that have also relied on observational research. Spurious precision has dramatic consequences not only for meta-analysis methodology and structural models calibrated using meta-analyses but also for the entire practice of evidencebased policy.

Project team of FSV UK (6)

doc. PhDr. Zuzana Havránková, Ph.D.

Princiapl investigator

+420 222 112 309

zuzana.havrankova@fsv.cuni.cz

Dr. Chishio Furukawa

Team member

64634870@fsv.cuni.cz

Dr. Heiko Jürgen Rachinger

Team member

heiko.jurgen.rachinger@fsv.cuni.cz

Pedro Bom, Ph.D.

Team member

Mohammad Ali Elminejad Anjileh, M.Sc., Ph.D.

Team member

m.ali.elminejad@gmail.com  |  ali.elminejad@fsv.cuni.cz

Mgr. Olesia Zeynalova, Ph.D.

Team member

olesia.zeynalova@fsv.cuni.cz

+3 more team members Show all team members