Wanyun Shao, Ph.D
  • Home
  • News
  • Bio
  • Research
  • Publications
  • Public Writing
  • Blog
  • Teaching
  • Contact
  • Links
  • Home
  • News
  • Bio
  • Research
  • Publications
  • Public Writing
  • Blog
  • Teaching
  • Contact
  • Links
Search by typing & pressing enter

YOUR CART

10/20/2017 0 Comments

How to detect signals from an expanding sea of noises

      There has been a replication crisis in medical and psychological studies in recent years. This NYT piece documents a prime example in social psychology. At the center of the "drama" are Amy Cuddy whose TED presentation on power pose has attracted tens of millions of views on YouTube and several statistically savvy researchers including Andrew Gelman.

      For many empirical scientists, using statistical analyses of a random sample to make inference for the entire population is the most reliable way to reveal some hidden patterns, except when a "pattern" already exists in the researcher's mind before even conducting the study. What the researcher needs to do is to find an ideal "path" to use the data to prove the preexisting pattern. This has been known as "p-hacking." (also read here, here, here,) Conducting research has never been easier. Scientists are "blessed" with so much data. With the blessing also comes a daunting task: correctly detecting signals from an expanding sea of noises. We empirical scientists thus have the obligation to constantly educate ourselves about the most recent advances in statistical methods. In the meantime, we need to constantly remind ourselves of putting aside our biases and wishful thinking while conducting research. Actions need to be taken at not only individual level but also collective level. Too much hype has been given to the so-called statistically significant findings. The glory of "statistical significance" has permeated the entire culture of science. Scientists are first and foremost humans who are driven by desires for success, fame, status, and respect. No one would pay much attention to a study that produces largely insignificant results. Maybe, incentives can be provided to researchers who after painstaking research design, meticulous data collection, and rigorous statistical tests end up with statistically insignificant results. I remember having an in-class discussion in graduate school. The professor told us that sometimes insignificant results can be meaningful too. Journal editors may want to give equal consideration to those rigorous studies that fail to produce significant results. While everyone is busy conducting original studies, incentives need to be provided to encourage replications. Only when we put checks and balances in place can we make this system more transparent and healthier.

         Scientists have started brainstorming about a path forward in the wake of the replication crisis. Some suggest, if P-hacking is such a rampant issue, why not make it harder to achieve significance? This Nature Human Behavior article proposes raising the bar by lowering the P-value threshold from 0.05 to 0.005.

0 Comments



Leave a Reply.

    Picture


    Wanyun Shao, Ph.D

    I am a geographer who studies risk decision making within a geographic context.

    Subscribe to Newsletter

    Categories

    All Climate Change Coastal Issues Community Resilience COVID 19 COVID-19 Drought Earthquake Energy Environmental Policy Flooding Geography GIS Hurricanes Natural Disaster Other Paris Accord Public Health Reading Risk Analysis Science Sea Level Rise Weather And Climate

    Archives

    February 2023
    January 2023
    August 2022
    April 2022
    March 2022
    February 2022
    December 2021
    November 2021
    October 2021
    September 2021
    July 2021
    June 2021
    April 2021
    February 2021
    December 2020
    October 2020
    July 2020
    December 2019
    October 2019
    September 2019
    August 2019
    June 2019
    April 2019
    March 2019
    February 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016

    View my profile on LinkedIn

    RSS Feed

Proudly powered by Weebly