Discrepancy between main statistics and post hoc ANOVA1rm

[eleonoraam000]

Hi Todd,

I am running a nonparametric inference with ANOVA1rm with 3 conditions repeated for each subject. The main statistics is not significant (attached: ex).

However, the post hoc corrected for multiple comparisons are significant between 2 conditions (attached:ex 2).

Can you explain why I see this discrepancy? Does it have to do, perhaps, with one of the curve from one conditions being largely variable?

Thanks, Eleonora

[0todd0000]

In general post hoc results are not guaranteed to precisely match main ANOVA results, even when the data are roughly normal and reasonably well-behaved. This is why there are several options for post hoc analysis, even for the case of simple 0D scalars. This problem can be amplified when going from 0D to 1D analysis and also when there are non-ideal data characteristics like non-normality, small samples, correlated observations, etc.

In this particular case I am not sure what is causing the discrepancy. It could be any of the following:

• small sample size
• inconsistent residual smoothness across conditions
• inconsistent variance across conditions
• outliers
• small number of permutations (iterations) in nonparametric inference
• etc.

If you attach a script I can check it for problems. If there are no coding problems then that would suggest that the discrepancy stems from characteristics of the data.