Multiple testing problem

[yossy-biomech]

I have a question about multiple testing problems. For example, in SPM analysis with a pair of t-tests, 101 t-tests are performed on 101 time-series change data. In other words, the test is repeated multiple times. In statistics, there are multiple testing problems. Is it okay to repeat the test with SPM?

[0todd0000]

Just in case you didn't mean to close this issue, here is a brief response:

There is actually just one test:

• Yes, the t statistic (or other test statistic value) is indeed computed at each point along the 1D domain.
• However, calculating the test statistic value at each point does not mean that one test is conducted at each point. Tests are only conducted when one calculates probabilistic quantities like critical thresholds. Calculating the 1D test statistic is just like calculating the 1D mean, or the 1D standard deviation; no tests are conducted when calculating these quantities.
• After the test statistic continuum is calculated, SPM conducts just one test, and this results in a single critical threshold; if the continuum crosses the threshold, then the null hypothesis is rejected.
• The SPM critical threshold is calculated based on the data's smoothness; the smoother the data are, the lower the threshold. If the data are infinitely smooth (i.e., a straight horizontal line), then the SPM critical threshold converges to the threshold obtained for standard distributions like Student's t distribution. If the data are infinitely rough (i.e., totally uncorrelated), then the SPM critical threshold converges to the Bonferroni correction across all 1D domain points.
• If there are N=100 points, you could interpolate the data to N=1000 or N=1,000,000 points, and the SPM critical will remain stable, within numerical tolerance. This is because the SPM threshold is calculated based on the data smoothness with respect to the domain size. In contrast, a Bonferroni-corrected threshold would increase monotonically with N.
• Thus SPM's single test (i.e., single critical threshold calculation) can be regarded as a smoothness-based correction for multiple comparisons across all domain points.