Dean McKay & Scott O. Lilienfeld
Dr. Jeffrey Lieberman of Columbia University, Past President of the American Psychiatric Association, recently took to Twitter to proclaim that psychology is in dire straits (https://twitter.com/DrJlieberman/status/637085819505737728). Dr. Lieberman pointed to the recent evidence, published in the prestigious journal Science, revealing that many social and cognitive psychology studies have failed to replicate (http://www.sciencemag.org/content/349/6251/aac4716.full). These recent findings prompted Dr. Lieberman to conclude that psychology is “in shambles.” In addition, Dr. Lieberman referred to an op-ed in the New York Times as ‘lame,” largely because the author suggested that psychology’s problems with replication were scarcely different from those of other sciences (see op-ed here: http://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html?ref=opinion&_r=0 and see Dr. Lieberman’s tweet here: https://mobile.twitter.com/DrJlieberman/status/638654836842430465).
SSCP’s take on these issues is rather different.
To be sure, psychology has its challenges with replicability, and the recent Science article underscores them. As the authors of the Science article note, however, their findings do not mean that the original findings were incorrect (although they have been widely misinterpreted in this fashion). They mean only that we should place less stock in the initial findings than we typically have, and that we will need to await further replication efforts – and ultimately meta-analyses – before sorting out which results are genuine and which are not. The bottom line is that we need to change the way we commonly think about research in psychology, psychiatry, and allied disciplines. Each new finding needs to be viewed not as anything remotely close to the final word, but rather as one piece of a large puzzle that will eventually be filled in.
Just as important, there’s ample reason to believe that replication difficulties are not unique to psychology. Despite its notable achievements in alleviating the suffering of individuals with mental illness, psychiatry too has struggled with its share of replication problems, such as those regarding the efficacy of transcranial magnetic stimulation for depression (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2993526/), the search for dependable biomarkers for psychopathology (http://www.nature.com/mp/journal/v17/n12/abs/mp2012105a.html), and the trustworthiness of most candidate gene studies of psychopathology, (http://ajp.psychiatryonline.org/doi/abs/10.1176/appi.ajp.2011.11020191). More broadly, John Ioannidis’ now famous analyses point to rampant problems with replicability in medicine, including psychiatry (http://jama.jamanetwork.com/article.aspx?articleid=201218), with many initially reported effects failing to hold up in later studies. There are even reasons to doubt that findings in many domains of psychology are less replicable than those in physics, such as particle physics (see http://psycnet.apa.org/journals/amp/42/5/443/). When it comes to replicability challenges, we’re all in this together.
One essential point missing from Dr. Lieberman’s communications is that the field of psychology has taken the lead in highlighting difficulties with replicability (http://pps.sagepub.com/content/7/6/528.short). Moreover, psychology - arguably more than any domain of science - has undertaken concerted efforts to estimate the magnitude of the replicability problem and to develop methodological safeguards against it. Such painstaking and at times painful self-examination is a sign of scientific strength, not weakness. Indeed, the beauty of psychological science has been its longstanding capacity to use psychological science to enhance the quality of psychological science. To the extent that we identify endemic shortcomings with some of our methodologies, we will do so by using science – and we will use science to root them out.
Those of us in psychological science certainly have our work cut out for us with respect to replicability. Fortunately, by engaging in rigorous self-scrutiny (as demonstrated by the recent article in Science), psychology is taking the initiative in attempting to confront this problem and place its profession on firming scientific footing. We very much hope that Dr. Lieberman will encourage his colleagues in psychiatry to join forces with psychology and follow suit.
