Difference in inference between 1) two way mixed Anova vs 2) two sample t-test on change

[bernard-liew]

Dear Todd,

How have you been?

I did some analysis on a data using a two group pre-post design. There are 14 people group 1 and 16 in group 2. So I did the most logical test and that is the repeated measures ANOVA with one repeated factor. When I look at the interaction SPM FABi, there were no significance. But when I did a two sample t test on change (post-pre) waveforms, there was a significance.

I read from (http://pareonline.net/pdf/v14n6.pdf), that "the most relevant F is for the treatment-by-time interaction, and it turns out that it is mathematically equivalent to the square of the t for the gain scores." When I look at the SPM maps, yes it reflects this. Hence, I think the reason for the difference in inference is the critical threshold. It is much lower for the t-test change scores, than for the anova. Could you help me with the reasons? Or some links to some materials? I just hope to get a more nuanced understanding.

Many thanks for the ongoing assistance.

Kind regards, Bernard

[0todd0000]

Hi Bernard,

Your interpretations of the relation between the F and t statistics are correct:

• The F continuum is equal to the squared t continuum
• The critical F value is equal to the squared critical t value.

However, instead of two-sample tests try paired t tests. Paired and not two-sample tests should be used for post hoc analysis of repeated measure (RM) factors.

Please let me know if using paired post hoc tests solves the problem.

Todd

[bernard-liew]

Dear Todd,

Many thanks for getting back to me. I have followed your advice and it does seem to help me understand. However, the problem isn't so much as it pertains to posthoc. It also boils down to the primary level test choice. As you can see, I have attached three possible test 1) mixed Anova, 2) 2 sample t test on change scores, 3) paired t test for each group, and 4) knee flexion angle

My understanding (naively) is that when one performs a mixed anova, the number one point of interest is if the change in group 1 is different from change in group 2. This means a two sample t test on change scores is the logical alternative to mixed-anova (as a primary test).

Since the critical threshold is calculated from field smoothness, could it be that the difference in results be due to different smoothness of the raw waveforms and change waveforms?

Many thanks for the kind assistance.

Regards, Bernard

[0todd0000]

Hi Bernard,

I just realized that I forgot to reply to your comment, I'm very sorry for the delay!!

Since the critical threshold is calculated from field smoothness, could it be that the difference in results be due to different smoothness of the raw waveforms and change waveforms?

The critical threshold depends on both the estimated field smoothness and the degrees of freedom (DF), so making changes that affect either will change the threshold. Changing the experimental design changes the residuals, which in turn changes the smoothness estimate. An experimental design change may also change the degrees of freedom. If the design doesn't drastically change the smoothness estimates should be relatively stable. In the results above it looks like the critical thresholds are all close to t = 2.5, implying that the smoothness estimates and degrees of freedom are fairly stable. So in this case I don't see a big change in critical thresholds across the three results.

Todd

[bernard-liew]

Dear Todd,

Not a problem for the delay, and thanks for the information.

On a side note, this may not be appropriate for this thread, but a peer interested in spm1d, but presently a user of Octave was wondering if the matlab's version can be used on that platform.

Kind regards, Bernard

[0todd0000]

Hi Bernard, I don't think spm1d will work in Octave because a fair bit of spm1d is written using MATLAB-style OOP, which Octave doesn't support as far as I know. Todd