search

trevismd / statannotations

add statistical significance annotations on seaborn plots. Further development of statannot, with bugfixes, new features, and a different API.
Other
619 stars 67 forks source link

What does * (ns) mean? #129

Closed JohannesWiesner closed 9 months ago

JohannesWiesner commented 10 months ago

I ran multiple comparison corrections using Benjamini-Hochberg procedure. Some of my tests have a * (ns) symbol. Wonder what this means? It does also not appear in the table that is printed to console before running the tests:

p-value annotation legend:
      ns: p <= 1.00e+00
       *: 1.00e-02 < p <= 5.00e-02
      **: 1.00e-03 < p <= 1.00e-02
     ***: 1.00e-04 < p <= 1.00e-03
    ****: p <= 1.00e-04
trevismd commented 9 months ago

Sorry about the delay @JohannesWiesner ...

It means that due to the correction method applied, your "*" which means between .01 and .05 is actually statistically not significant (ns). You can see the correction_formatargument of Annotator.configure for changing this rendering (for example to simply ns with .configure(correction_format="replace"). You'll just have to be mindful that the legend will not be accurate anymore (but you see the (ns) in the statistical test result).

JohannesWiesner commented 9 months ago

But wouldn't it be better if the * would just completely disappear by default if it doesn't survive the multiple comparisons correction?

trevismd commented 9 months ago

Opened a discussion around this suggestion: https://github.com/trevismd/statannotations/discussions/137 (Trying to progressively clean up the issues vs howtos vs why-dont-we ;))