Clarification regarding theory behind SPM1D paired t-test

[itbellix]

Dear @0todd0000, thank you for providing such a powerful tool, and for clarifying so many of the doubts its users have. I am currently trying to interpret how SPM1D works, in particular in the context of a paired t-test. Reading in the literature, I found the following statement (taken from the appendix of one of your papers at https://doi.org/10.1016/j.gaitpost.2014.01.023):

Since SPM results contain a test statistic value at each point in the trajectory, one may be tempted to infer that a separate statistical test has been conducted at each node. In fact, SPM conducts just one test.

I am a bit confused regarding this statement, as what I understand is that the t-statistic (SPM{t}) is computed at each point of the trajectory, and the resulting time-series of the values of SMP{t} is continuous as the original data being compared are 1D-continua. Assuming that the 1D-continua that are being compared are a function of time, is not clear to me is how (and if) the SPM retains a temporal correlation in the statistical analysis. I would say this has to do not with the SPM{t} itself, which is evaluated "statically" at each point (with the formula in https://github.com/0todd0000/spm1d/issues/90#issue-383882220), but it is also not related to the value of t (the threshold that the SPM{t} metric has to exceed), which is a single scalar value. So, my question is: how does SPM differ from performing a paired t-test at each point in the trajectory? I am guessing that information about how the means and standard deviations of the two population vary along the continuum are lumped into the actual value of t, is this indeed the case?

Thank you very much and have a good day, Italo

[0todd0000]

how does SPM differ from performing a paired t-test at each point in the trajectory?

The SPM result is invariant to the number of points Q used to sample the continuum, provided Q is sufficiently large to exceed the Nyquist sampling criterion. The same cannot be said of independent tests conducted at each point because corrected thresholds over these Q tests (e.g. Bonferroni thresholds) are directly dependent on Q.

Presuming that Q=100 is sufficiently large to contain the frequency content of interest, one could resample the data at Q=1000, Q=1e6 or even more points and the SPM results (critical threshold and p-values) will not change (or will change negligibly). Contrastingly, Bonferroni and other similar results will change dramatically as a function of Q, with critical thresholds rising to infinity and probabilities rising to 1 as Q becomes large.

I am guessing that information about how the means and standard deviations of the two population vary along the continuum are lumped into the actual value of t*, is this indeed the case?

Yes. For a common test like a paired test, the t statistic is defined the same way as in a standard test. If you were to use scipy.stats.ttest_rel, for example, to calculate the t-value at each of the Q points, these t-values should be exactly the same as those that spm1d.stats.ttest_paired yields.

The key difference is that scipy.stats.ttest_rel will have conducted statistical inference Q separate times (i.e., calculate hypothesis-relevant probabilities Q separate times). SPM conducts inference just once.

[itbellix]

Dear @0todd0000, thank you very much for your answer. Then the t computation is really what distinguishes SPM results from independent tests. As a follow-up question, I have been looking in some papers but did not find an explicit formulation how t is computed. Could you maybe point me at where to find it? Thank you again.

Cheers, Italo

[0todd0000]

I have been looking in some papers but did not find an explicit formulation how t* is computed. Could you maybe point me at where to find it?

See Eqn. 3 in this paper. A free preprint version of this article is available on Research Gate.

[itbellix]

Great, thank you!