0todd0000 / spm1d

One-Dimensional Statistical Parametric Mapping in Python
GNU General Public License v3.0
61 stars 21 forks source link

Paired t-tests #113

Closed burniel closed 4 years ago

burniel commented 4 years ago

I have used SPM paired t-tests to compare within-participant biomechanical variables, and set alpha to 0.05. I have had quesiton raised by a reviewer of my manuscript on SPM which I would be grateful for your help with answering. They believe the standard threshold value for SPM by Pataky’s SPM1d software is a t or z value of 3.0. If a value in this range is used, it means that only points along the curve that have a p-value of < 0.0014 are being considered (assuming a z-distribution). Is this correct? As I thought the critical random field theory threshold was calculated within SPM for each data set or have I misunderstood?

0todd0000 commented 4 years ago

Your interpretation is correct. The threshold is indeed calculated separately for each dataset.

SPM uses Gaussian random field theory (RFT) to calculate the threshold that only alpha% of equivalently smooth Gaussian random fields would cross, when the null hypothesis is true. The RFT threshold changes with alpha, sample size and data smoothness.

The easiest way to think of this is with the following pseudo-code:

For usual t tests (on 0D daa):

threshold_0D_data = students_t_function(alpha, degrees_of_freedom)

For SPM t tests (on 1D data):

threshold_1D_data = rft_function(alpha, degrees_of_freedom, smoothness)

where "smoothness" is estimated as the FWHM (see Figure below). When the data are rough, they are more likely to cross an arbitrary threshold, so the threshold increases. As the data become smooth, and approach infinite smoothness, the 1D threshold approaches the 0D the threshold.

In your response to reviewers, if you'd like to cite the literature instead of this discussion, I'd suggest the following paper, which numerically validates a variety of RFT probabilities for 1D data, including showing that RFT thresholds accurately control alpha=0.05:

Pataky TC (2016) rft1d: Smooth One-Dimensional Random Field Upcrossing Probabilities in Python. Journal of Statistical Software 71 : i07. http://dx.doi.org/10.18637/jss.v071.i07

burniel commented 4 years ago

Thanks Todd, this is really helpful.