Events

Abstract:  Social scientists face a dual problem of methodological abundance and model uncertainty. There are many ways to conduct an analysis, but the true model is unknown. Multiverse analysis addresses this challenge by recognizing ‘many worlds’ of modeling assumptions, using computational power to reveal a large set of plausible estimates. We advance new methods for the functional form multiverse, considering OLS, logit, Poisson, inverse probability weighting, and two forms of matching. How much do empirical results depend on the choice of functional form? Are some functional forms more robust – stable across the choice of controls – than others? Our multiverse estimator takes all plausible combinations of control variables and functional forms, yielding a distribution of estimates that the data can support under alternative assumptions. We apply this to three empirical cases, estimating thousands of specifications: the effect of unemployment on wellbeing, the role of education in voting for Donald Trump, and discrimination by skin tone in professional soccer. We also compare computational multiverse analysis to a ‘many analysts’ crowdsourcing project. In our cases, we find that (1) most functional forms generate similar estimates on average, but differ by estimate stability; (2) no functional form is more stable across sets of controls than OLS; (3) matching performs especially poorly in terms of stability; and (4) crowdsourcing generates a wider range of estimates than a comparable functional form multiverse.   

The Sociology Seminar Series for Hilary Term is convened by Richard Breen and Janne Jonsson. For more information about this or any of the seminars in the series, please contact sociology.secretary@nuffield.ox.ac.uk.