0todd0000
commented
8 years ago Hi Nirav,
Excellent questions.
-
Yes, precise p values for p > alpha can indeed be computed, but there are two caveats. (Caveat 1) Since there are no supra-threshold clusters the p value would pertain only to the maximum test statistic value across the whole 1D continuum. (Caveat 2) Random Field Theory (RFT) solutions are generally validated only for low probabilities (usually p<0.4 or p<0.3). The RFT solutions may indeed be valid for all probabilities between 0 and 1, but since they generally have not been validated a convention of "p > alpha" emerged in the SPM literature. Both caveats aside, the p values we're talking about can roughly be computed using the rft1d.teststat.sf functions like this:
import rft1d maximum_f_value = 5.0 #maximum f value across the 1D continuum df = (2, 14) #degrees of freedom nodes = 101 #number of continuum nodes fwhm = 10.0 #1D smoothness p_value = rft1d.f.sf(maximum_f_value, df, nodes, fwhm) #in this case: p = 0.391Note that, while the rft1d.teststat.sf results are approximate, I've done some partial validations of large p values using t statistics, and they generally seem quite good, with perhaps some minor discrepancies for very large p values like p=0.9. In the next version of spm1d we'll try to include some more details about this, preferably with validated p values when p > alpha.
For now, if you wish to report these p values in a paper, I'd recommend being cautious. For completeness you could write something like: "The null hypothesis was not rejected at alpha=0.05, and the probability that 1D Gaussian continua with the same smoothness would produce an F continuum with the same maximum value is approximately p = 0.391. This solution is approximate pending numerical validations of large p values (Friston et al. 2007) which have not yet appeared in the literature."
- I think the best way is with unfortunately with the SPM graphs. The fundamental problem with reporting F or t values, and even with reporting Cohen's d, is that sample sizes (degrees of freedom) curb their probabilistic meaning. Thus t=3.0 in one dataset may be probabilistically equivalent to t=4.0 in another dataset. With the SPM graphs the critical threshold anchors the results in probability theory, where the critical threshold has the identical probabilistic meaning across all experiments. That being said, I think all of the possibilities you mention are fine provided they are used for comparing datasets ONLY when the degrees of freedom are identical or at least quite similar.
As an aside, I fully appreciate the problem of how to describe the results. My main advice, based on the Neuroimaging literature, is that SPM results transcend the convention of text descriptions to a certain extent because the graphs depict results that text descriptions can never fully describe. In the Neuroimaging literature effect sizes and most other statistics are reported at the continuum level, in that case as a 3D continuum within the brain, and these are described in the text qualitatively, often augmented with cluster-specific p values.
Cheers,
Todd
Kneerav
Hi Todd,
Just had a couple of questions:
a) Take the peak F or T statistic for a given cluster b) Take the average F or T statistics for a given cluster c) Report the F or T range (the lower end being the critical threshold, upper end being the peak) d) Compute a paired Cohen's d (for my post-hoc paired t-tests) for each node across the waveform, and then take either the peak, average, or range (within the suprathreshold cluster period) as described previously.
Is there any particular method you see as the most valid and/or acceptable?
Cheers, Nirav