Abstract

The replication crisis in science has not spared functional magnetic resonance imaging (fMRI) research. A range of issues including insufficient control of false positives, code bugs, concern regarding generalizability and replicability of findings, inadequate characterization of physiological confounds, over-mining of repository datasets, and the small sample sizes/low power of many early studies have led to hearty debate in both the field and the press about the usefulness and viability of fMRI. Others still see enormous potential for fMRI in diagnosing conditions that do not otherwise lend themselves to non-invasive biological measurement, from chronic pain to neurological and psychiatric illness. How do we reconcile the limitations of fMRI with the hype over its potential? Despite many papers hailed by the press as the nail in the coffin for fMRI, from the dead salmon incident of 2009 to cluster failure more recently, funders, researchers, and the general public do not seem to have reduced their appetite for pictures of brain maps, or gadgets with the word “neuro” in the name. Multiple blogs exist for the sole purpose of criticizing such enterprise. The replicability crisis should certainly give ‘neuroimagers’ pause, and reason to soul-search. It is more important than ever to clarify when fMRI is and when it is not useful. The method remains the best noninvasive imaging tool for many research questions, however imperfect and imprecise it may be. However, to address past limitations, I argue neuroimaging researchers planning future studies need to consider the following five factors: power/effect size, design optimization, replicability, physiological confounds, and data sharing.

Keywords: fMRI, open science

References

Woo CW, Krishnan A, Wager TD. Cluster-extent based thresholding in fMRIanalyses: pitfalls and recommendations. Neuroimage. 2014 May;91:412-19. Available from: https://doi.org/10.1016/j.neuroimage.2013.12.058

Eklund A, Nichols TE, Knutsson H. Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. Proc Natl Acad Sci USA. 2016 Jul;113(28):7900-05. Available from: https://doi.org/10.1073/pnas.1602413113

Cox RW, Chen G, Glen DR, Reynolds RC, Taylor PA. fMRI clustering in AFNI: false-positive rates redux. Brain Connect. 2017 Apr;7(3):152-71. Available from: https://doi.org/10.1089/brain.2016.0475

Drysdale AT, Grosenick L, Downar J, Dunlop K, Mansouri F, Meng Y, et al. Resting-state connectivity biomarkers define neurophysiological subtypes of depression. Nat Med. 2016 Dec;23(1):28-38. Available from: https://doi.org/10.1038/nm.4246

Dinga R, Schmaal L, Penninx BWJH, van Tol MJ, Veltman DJ, van Velzen L, et al. Evaluating the evidence for biotypes of depression: methodological replication and extension of Drysdale et al. Neuroimage Clin. 2019;22:101796. Available from: https://doi.org/10.1016/j.nicl.2019.101796

Dinga R, Schmaal L, Marquand AF. A closer look at depression biotypes: correspondence relating to Grosenick et al. (2019). Biol Psychiatry: Cog Neurosci Neuroimaging. 2020 May;5:554-5. Available from: https://doi.org/10.1016/j.bpsc.2019.09.011

Grosenick L, Liston C. Reply to: a closer look at depression biotypes: correspondence relating to Grosenick et al. (2019). Biol Psychiatry: Cog Neurosci Neuroimaging. 2020;5(5):556. Available from: https://doi.org/10.1016/j.bpsc.2019.11.002

Tsvetanov KA, Henson RNA, Rowe JB. Separating vascular and neuronal effects of age on fMRI BOLD signals. Philos Trans R Soc Lond B Biol Sci . 2020 Nov;376:20190613. Available from: https://doi.org/10.1098/rstb.2019.0631

Özbay PS, Chang C, Picchioni D, Mandelkow H, Moehlman TM, Chappel-Farley MG, et al. Contribution of systemic vascular effects to fMRI activity in white matter. Neuroimage. 2018 Aug;176:541-9. Available from: https://doi.org/10.1016/j.neuroimage.2018.04.045

Thompson WH, Wright J, Bissett PG, Poldrack RA. Dataset decay and the problem of sequential analyses on open datasets. ELife. 2020 May;9:e53498. Available from: https://doi.org/10.7554/eLife.53498

Stockton N. 2017 June. Don't be so quick to flush 15 years of brain scan studies. Iowa USA: Wired; 2016 Jul 8. Available from: https://www.wired.com/2016/07/dont-quick-flush-15-years-brain-scan-studies/

Morris E. Why we need guidelines for brain scan data. Iowa USA: Wired; 2019 Sept 17. Available from: https://www.wired.com/story/why-we-need-brain-scan-d ata-guidelines/

D’Esposito M. 2019. Are individual differences in human brain organization measured with functional MRI meaningful? Proc Natl Acad Sci USA. 2019 Nov;116(45):22432-4. Available from: https://doi.org/10.1073/pnas.1915982116

Bennett CM, Miller MB, Wolford GL. Neural correlates of interspecies perspective taking in the post-mortem atlantic salmon: an argument for multiple comparisons correction. Neuroimage. 2009 J ul;47(1):S125.

Cremers HR, Wager TD, Yarkoni T. 2017. The relation between statistical power and inference in fMRI. PLoS One. 2017 Nov;12(11):e0184923. Available from: https://doi.org/10.1371/journal.pone.0184923

Mills-Finnerty C, Hanson C, Khadr M, Hanson S.J. Computations and connectivity underlying aversive counterfactuals. Brain Connect. 2020 Nov;10(9):467-78. Available from: https://doi.org/10.1089/brain.2020.0766

Miyakawa T. No raw data, no science: another possible source of the reproducibility crisis. Mol Brain. 2020 Feb;13:24. Available from: https://doi.org/10.1186/s13041-020-0552-2

Chambers C. What’s next for registered reports? Nature. Sept;573:187-9. Available from: https://doi.org/ 10.1038/d41586-019-02674-6

Yarkoni T, Poldrack R, Nichols T, Van Essen D, Wager T. NeuroSynth: a new platform for large-scale automated synthesis of human functional neuroimaging data. Front Neuroinform Conference Abstract: 4th INCF Congress of Neuroinformatics. 2011 Sept. Available from: https://doi.org/10.3389/conf.fninf.2011.08.00058

Bowring A, Maumet C, Nichols TE. Erratum: exploring the impact of analysis software on task fMRI results. Hum Brain Mapp. 2020 Dec;42(5):1564-78. Available from: https://doi.org/10.1002/hbm.25302

Murphy K, Birn RM, Bandettini PA. 2013. Resting-state fMRI confounds and cleanup. Neuroimage. 2013 Oct;80:349-59. Available from: https://doi.org/10.1016/j.neuroimage.2013.04.001

Park HD, Barnoud C, Trang H, Kannape OA, Schaller K, Blanke O. 2020. Breathing is coupled with voluntary action and the cortical readiness potential. Nat Commun. Feb;11:289. Available from: https://doi.org/10.1038/s41467-019-13967-9

Mills-Finnerty C. Barriers to reproducibility: misalignment of career incentives and open science best practices. J Rep Neurosci . 2020 Aug;10. Available from: https://doi.org/10.31885/jrn.1.2020.304