0todd0000 / spm1d

One-Dimensional Statistical Parametric Mapping in Python
GNU General Public License v3.0
61 stars 21 forks source link

Bonferonni Corrections #170

Closed meechan1983 closed 3 years ago

meechan1983 commented 3 years ago

Hi Todd

I have started playing with SPM for a project I am working on comparing the effect of intensity. We ran an anova (40%,60%,80%,100%,120%,140%) and then multiple paired t tests independently ( 40vs60, 40vs80 and so on [ this results in a total of 15 load comparisons ). This was done by copying and pasting each load into variableA and variableB

Would I need to Bonferonni correct this? if so, how would I could about doing this? I have attached the code

One paper I read in JSCR did alpha/12 and another recent paper stated 'SPM has multiple comparison correction built in'

One more question if possible, to check where the shaded differences occur, I used the brush function to drag over it and visually inspect, is there a better way of doing this?

Many thanks in advance

%(1) Compute confidence intervals: alpha = 0.05; ci2 = spm1d.stats.ci_pairedsample(variableA, variableB, alpha, 'meanA', 'tailB'); disp(ci2)

%(2) Plot: close all figure('position', [0 0 1000 300]) myxlim = [0 size(variableA,2)-1];

% plot hypothesis test: subplot(122) spmi = spm1d.stats.ttest_paired(variableA, variableB).inference(alpha); spmi.plot(); spmi.plot_threshold_label(); spmi.plot_p_values(); xlabel('Normalised Movement Time (%)' ,'FontSize',10,'FontWeight','bold','Color','k');

%plot CI (possibility 3) subplot(121) ci2.plot() xlim(myxlim) title('Mean and 95% CI') xlabel('Normalised Movement Time (%)' ,'FontSize',10,'FontWeight','bold','Color','k'); ylabel('Velocity m/s)' ,'FontSize',10,'FontWeight','bold','Color','k');

0todd0000 commented 3 years ago

Would I need to Bonferonni correct this? if so, how would I could about doing this?

A multiple comparisons correction is indeed necessary, but not necessarily the Bonferroni correction. If you use the Bonferroni, you could use it like this:

ntests = 12
p_critical = spm1d.util.p_critical_bonf(alpha, ntests)
posthoc_results = spm1d.stats.ttest_paired(variableA, variableB).inference(p_critical)

Note that the Bonferroni correction is a conservative approximation and does not necessarily yield results that are fully consistent with the original ANOVA results.



One paper I read in JSCR did alpha/12 and another recent paper stated 'SPM has multiple comparison correction built in'

The built-in multiple comparisons correction is the random field theory (RFT) correction, which corrects across the 1D domain (usually time).



One more question if possible, to check where the shaded differences occur, I used the brush function to drag over it and visually inspect, is there a better way of doing this?

Yes, use the Clusters class, like this:

alpha = 0.05
t     = spm1d.stats.ttest(Y, mu)
ti    = t.inference(alpha, two_tailed=False, interp=True, circular=False)
print( ti.clusters )

which yields something like this:

[Cluster
    threshold       :  3.227
    centroid        :  (75.014, 3.535)
    isinterpolated  :  True
    iswrapped       :  False
    endpoints       :  (72.822, 77.247)
    extent          :  4.425
    extent (resels) :  0.16667
    height (min)    :  3.22704
    P               :  0.04441
 ]

The endpoints attribute contains each cluster's endpoints along the 1D domain.

meechan1983 commented 3 years ago

Hi Todd

Many thanks.

Is it also possible to divide the alpha by the number of comparisons and use that as the reduced threshold? (which I believe those authors did)

I'll have a go and see if I can figure it out

Best regards

David

0todd0000 commented 3 years ago

Yes, dividing alpha by the number of comparisons is very close to the Bonferroni correction. For example:

alpha             = 0.05
ntests            = 8
p_critical_approx = alpha / ntests    # this is 0.00625
p_critical_bonf   = spm1d.util.p_critical_bonf(alpha, ntests)   # this is 0.00639
meechan1983 commented 3 years ago

thanks todd