0todd0000
commented
8 years ago Hi Teresa,
The Bonferroni correction is indeed conservative and may not yield expected results in some cases.
Tukey’s post hoc analyses are possible but they are not yet implemented in spm1d. There was a discussion about this topic in the spm1d Python forum: https://github.com/0todd0000/spm1d/issues/6
Tukey's procedure can be implemented, as can other post hoc procedures, but please note that this will require you to get into the lower-level probability functions, especially ./spm1d/spm8/spm_P_RF.m, and then also to validate the procedure through random field simulation.
The easiest alternative post hoc procedure to implement and validate is probably the Holm-Bonferroni correction, which only requires p values. https://en.wikipedia.org/wiki/Holm–Bonferroni_method Note that the cluster-level p values reported in the SPM structure are not appropriate because there are no cluster-level p values if the t continuum does not cross the alpha-threshold. Instead you could use ./spm1d/spm8/spm_P_RF.m to compute one p value fore each test, based either on a set-level inference or on the maximum t value.
Todd
The following question was posed by email, I'm reposting here...
I need help. I recently started using the 2-way repeated measures ANVOA test in the Matlab package.
My design is 2x3 (within-within). I have a few variables that have significant interaction of the main effects but I am having trouble trying to adjust for my family wise error during the post-hoc paired Ttests. Theoretically there should be 15 comparisons but I will only be looking at 9 unconfounded comparisons. However, when using a bonferroni correction of alpha this reduces my alpha =0.05/9=0.0055 which I believe is way too conservative. I would like to use something more liberal, like a tukey's test but I am not sure how I can do this within the script or if it is even valid. Do you have a better suggestion on how I might be able to protect against this Type1 error?
Thank you very much, Teresa Flaxman