search

0todd0000 / spm1d

One-Dimensional Statistical Parametric Mapping in Python
GNU General Public License v3.0
61 stars 21 forks source link

Post-hocs two-way RM anova on 1 factor #172

Closed aurelienPatoz closed 3 years ago

aurelienPatoz commented 3 years ago

Dear Sir,

We have conducted a two-way (group x insole) RM anova using spm1d in python as

spm1d.stats.anova2onerm(data, groups_factor, insoles_factor, subjects, equal_var = True)

We have 2 groups, 3 different insoles, and 115 subjects.

Statistically significant group effect, insole effect, and interaction effect were reported.

Here is the insole effect: spm_force_Y_insole-effect.pdf

However, when I conduct the post-hoc analysis for the insole effect using

t = spm1d.stats.ttest2(data1, data2, equal_var = True)

and perform the inference with

t.inference(alpha = p_critical, two_tailed = True))

I do not get any significant difference.

Here are the 3 figures: spm_force_Y_no-group_11_hard-11_post-hoc.pdf spm_force_Y_no-group_11_soft-11_post-hoc.pdf spm_force_Y_no-group_soft-11_hard-11_post-hoc.pdf

Note that I have set p_critical using Bonferroni corrections assuming 3 post-hoc tests (because I have 3 conditions and I don't need to use the group appartenence when I am looking at the insole effect).

In addition, we have very small visible differences between our insoles as it can be noticed on this figure: force_y_no-group.pdf

Do we get a significant insole effect because of the very high number of participants?

Could you please let us know if there are any mistakes in our analysis and could you please help us with the understanding of our results?

Best regards, Aurélien

0todd0000 commented 3 years ago

However, when I conduct the post-hoc analysis for the insole effect... I do not get any significant difference.

Since the insole factor is a repeated measures (RM) factor, spm1d.stats.ttest_paired would be more appropriate for post hoc analysis. This may not yield perfect correspondence with the ANOVA results, but the correspondence should be qualitatively better than when using spm1d.stats.ttest2



In addition, we have very small visible differences between our insoles

Since the insole factor is an RM factor, it may be appropriate to plot pairwise differences. RM analyses test hypotheses regarding mean paired differences, and not group means, so plotting paired differences may yield results that are qualitatively more similar to the RM test results.



Do we get a significant insole effect because of the very high number of participants?

Possibly. To answer this question you'd need:

  1. an a priori hypothesis regarding the effect size you wish to detect during the experiment (e.g. an effect of 0.05 BW), then
  2. power analysis to calculate the appropriate sample size.

Using more subjects than this could indeed over-power the study.

aurelienPatoz commented 3 years ago

Thank you very much for having pointed the mistake. I have corrected my notebook and now it reflects the results of the anova.

Moreover, plotting the paired differences is also more reflecting of the post hoc results.

Best regards, Aurélien

0todd0000 commented 3 years ago

Great! I'll close this issue now, but please feel free to re-open it or submit a new issue if needed.