search

webermarcolivier / statannot

add statistical annotations (pvalue significance) on an existing boxplot generated by seaborn boxplot
MIT License
394 stars 74 forks source link

Show only significant comparisons #43

Closed smgroves closed 4 years ago

smgroves commented 4 years ago

Is there an easy way to show only the comparisons that end up as significant? I would like to compute all comparisons, but showing the non-significant ones make the plot very busy.

webermarcolivier commented 4 years ago

Computing all possible pairwise comparisons and selecting only significant differences must be handled in a multiple comparisons framework, like ANOVA, or multiple test corrections. We are currently implementing some basic features, but for the moment this problem lies outside the scope of this tool (see also #25).

One possible approach would be to use one-way ANOVA (if values are normally distributed) or Kruskal-Wallis test, followed by a post-hoc analysis to identify significant pairs by using Tukey test. Clearly, those should be performed prior to plotting the data on the boxplot, and thoughtfully chosen depending on the data characteristics and experiment design. Then, once the significant pairs have been identified, the annotation can be drawn by using custom p-values computed in the tests.

The latest version of the tool (development version, not pip package) (see #40) include the Bonferroni method for multiple comparisons correction, and the option is active by default (comparisons_correction='bonferroni'). When the option is active, the p-values are corrected by the number of comparisons. However, note that this correction is very conservative and some significant pairs could be missed. We are planning to include more advanced multiple comparisons methods such as Benjamini-Hochberg.

Nelson-Gon commented 4 years ago

Regarding this issue, is anova implemented in these tests?

CameraEngineer commented 3 years ago

In regards to @smgroves comment back in Feb, although it is great to display every combination specified the box_pairs list it is really nice to remove the lines that are not significant. Especially for publication figures, which is what I need. What I did using the code downloaded using version statannot 0.2.3 was add a simple print statement that indicated which groups I was comparing were not significant.

image

Then at the end of the file I had to allow a pass statement if there were not significant differences.

image

Hope that helps! Awesome job on this module. Truly a life saver.

SamStaps commented 3 years ago

Thank you @CameraEngineer for sharing your quick solution. Implemented this now. However, thanks @webermarcolivier for the careful considerations about multiple comparisons, and this awesome module.