Psychology's "replication crisis"

This week, I came across this article via fivethirtyeight. It discussed a recent crisis in the field of psychology--researchers being unable to replicate past experiments previously accepted within the discipline. The specific experiment the article discussed was one related to facial feedback and emotion (e.g. the idea that just the physical act of smiling can actually make you happier), but the discipline as a whole has been struggling immensely will recent failures to replicate past studies. It appears that issues of small sample sizes, inconclusive results, and manipulated statistical tactics have led to serious issues with formerly accepted literature. This recent crisis has led to several new initiatives designed to prevent these problems from happening in the future, such as the "Many Labs" projects (focused on replicating past studies) and the Psychological Science Accelerator (a collaborative network giving members access to 548 labs in 72 countries).

However, this is not just a problem confined to the field of psychology. Many other disciplines--both in the social sciences and "hard" sciences--have been criticized recently for issues of misreporting significant effect sizes ("p-hacking" or "selective reporting") and/or small sample sizes. For example, several researchers have recently critiqued the biological theory of menstrual synchrony, a study initially only run across 135 women, which has had mixed results when replicating.

I believe one of the reasons many disciplines have struggled recently with their own recent "replication crisis" is partially due to intense pressure at universities and research institutions to produce new literature. Academics in particular are constantly expected to turn out new studies with a previously untested hypothesis. And of course, successful hypotheses (or at least, rather, ones with shocking or previously unfounded results) receive way more funding and attention than those that are unsuccessful or just designed to confirm something previously "known" within the literature of a particular discipline. Grad students and professors are therefore incentivized (whether consciously or unconsciously) to skew their data from a recent hypothesis towards a favorable result. This leaves us with a pile of sometimes contradictory literature, making it difficult for individuals to decipher accurate information.

The recent recognition of this crisis, and push towards replication I believe to be a net positive for all academic disciplines. If universities/research institutions accepted failed hypothesis or those that simply confirmed/added to previously accepted literature the same way they do with new additions to a discipline, we might have less of an issue in sorting through thousands of published articles to find the reputable and replicable studies. Of course, sexy, new discoveries will always receive more excitement and attention than those that are a simple reaffirmation of existing knowledge. But if we don't soon shift our framing to also value studies that are less successful in their findings, we will continue to exacerbate the current replication crisis rather than solve it.


Popular Posts