search

0todd0000 / spm1d

One-Dimensional Statistical Parametric Mapping in Python
GNU General Public License v3.0
61 stars 21 forks source link

Non parametric tests parameters #68

Closed jabou356 closed 7 years ago

jabou356 commented 7 years ago

Hi Todd,

I am planning to use the non-parametric version of a paired t-test in the Matlab package and I had some questions:

1) Which method did you use for multiple comparisons? Is it the Single Threshold test proposed by Nichols & Holmes?

2) What is the statistic reported by the test? Are they simple t statistics or pseudo t-statistics as suggested in Nichols & Holmes? If pseudo t statistics are used, how is the variance smoothed across neiboring samples?

3) Is it possible to modify the parameters of the test (t-statistics vs pseudo-t, Single threshold test vs suprathreshold clusted tests, etc?)

Thanks!

Jason

0todd0000 commented 7 years ago

Hi Jason, great questions!

  1. Which method did you use for multiple comparisons? Is it the Single Threshold test proposed by Nichols & Holmes?

spm1d uses both the "single threshold" and "suprathreshold cluster" tests of Nichols and Holmes. The critical threshold calculation follows the former, and cluster-specific p value calculations follow the latter.

  1. What is the statistic reported by the test? Are they simple t statistics or pseudo t-statistics as suggested in Nichols & Holmes? If pseudo t statistics are used, how is the variance smoothed across neiboring samples?

Simple t statistics. Variance smoothing like in Nichols & Holmes can be quite useful when the measured variance field is quite rough, but in most biomechanics datasets I've seen the variance (and the t statistics) are quite smooth, so smoothing the variance field would be somewhat redundant. Note that the key benefit of NOT smoothing variance is that the test statistic fields for parametric and non-parametric procedures are identical. spm1d deliberately avoids variance smoothing so that differences between parametric and non-parametric tests arise purely from data non-normality.

  1. Is it possible to modify the parameters of the test (t-statistics vs pseudo-t, Single threshold test vs suprathreshold clusted tests, etc?)

Not in MATLAB. The Python version offers a "cluster_size" keyword option which you lets you specify the minimum cluster size for Nichols & Holmes' suprathreshold cluster test (and the same keyword can be used in parametric inference). This functionality is not implemented in the MATLAB version of spm1d. Nevertheless, customizations like the ones you describe are fairly straightforward to script in both Python and MATLAB. For example, please see the script "./examples/nonparam/0d/ex_ttest_standalone.m" which shows how one can conduct non-parametric inference, completely independent from spm1d. In there you could add arbitrary variance smoothing, arbitrary test statistic calculations, and arbitrary parameters for the "suprathreshold cluster" tests.

As an aside, there are currently no plans to extend spm1d's nonparametric options because there are simply too many possibilities for customizing non-parametric analysis. If the standard Nichols & Holmes approach (with simple test statistics) is not sufficient then I'd suggest manually scripting a custom non-parametric procedure.

Todd

jabou356 commented 7 years ago

Thanks a lot for the quick and clear response!

Jason