search

0todd0000 / spm1d

One-Dimensional Statistical Parametric Mapping in Python
GNU General Public License v3.0
61 stars 21 forks source link

Display P-value for nonsignificant differences #222

Closed menzejo closed 1 year ago

menzejo commented 2 years ago

Dear SPM experts

We would like to display the range of p-values for nonsignificant differences in a continuous comparison between two groups using the Mann Whitney U test. We were curious if there is an option to do this using SPM, in the graphs below for example in the range between 40° to 60° of shoulder abduction: Picture1

When we do see significant differences the P-value is automatically depicted as below: 2022-07-15_13-42-37

I am looking forward to your response!

Thank you and best regards Johanna Menze (johanna.menze@unibe.ch)

0todd0000 commented 2 years ago

There are two problems with displaying p-values greater than alpha:

  1. Cluster-level inference: these p-values are specific to clusters, as in the results above. A cluster exists only when alpha-defined threshold is traversed, so there are no clusters --- by definition --- when alpha is not traversed. So "non-significant" p-values do not exist for cluster-level inference.

  2. Point-level inference: it is indeed possible to calculate p-values for individual points, but (a) these p-values become inaccurate why p gets large (p > 0.4), and (b) those p-values are not directly comparable to cluster-level results because they are different probabilities.

I therefore recommend against reporting p-values when p > alpha, and instead reporting just "p > alpha". This follows standard SPM practice for reporting cluster-level probabilities. Will this be satisfactory?