Unexpected Post Hoc Results

[joshuatbland]

Hi. Hello. I have run a one-way repeated measures ANOVA on a dataset - I have included a plot of the ANOVA results and a plot for each subject. The ANOVA results suggest that there are significant within-subject differences between conditions along the length of the trajectory. However, post hoc tests do not show any significant results - this is very surprising. Any insights into what might be causing this would be greatly appreciated.

[0todd0000]

If you have used spm1d.stats.ttest2 for post hoc tests you may indeed see different results. If you use spm1d.stats.ttest_paired you should see post hoc results that more closely agree with these one-way RM-ANOVA results.

[joshuatbland]

We did indeed use the paired t-test for our post hoc tests but we did not get results that agree with the one-way RM-ANOVA results. Here our the results for the paired t-tests ( with Bonferroni correction):

[0todd0000]

Everything you've done sounds find, and I agree that these post hoc results are unexpected given the ANOVA results. To help me understand the analyses more completely can you please attach or copy-paste a script that generates the figures above?

[joshuatbland]

Yes, here is the code that was used to generate the figures:

%% One-Way Repeated Measures Anova 
%%--------------------------------------------------------------
[Y,A,SUBJ] = deal(ang,condition, specimen);

spm = spm1d.stats.anova1rm(Y, A, SUBJ);
spmi = spm.inference(0.05);

spmi.plot();
spmi.plot_threshold_label();
spmi.plot_p_values();

%% T-test Post-Hoc Tests 
%%--------------------------------------------------------------
alpha = 0.05
nTests = 3
p_critical = spm1d.util.p_critical_bonf(alpha, nTests)

% Condition 1 vs Condition 2
figure()
spm_one = spm1d.stats.ttest_paired(ang(cond1_indexes,:), ang(cond2_indexes,:));
spmi_one = spm_one.inference(p_critical, 'two_tailed', true, 'interp',true);
spmi_one.plot();

% Condition 1 vs Condition 3 
figure()
spm_two = spm1d.stats.ttest_paired(ang(cond1_indexes,:), ang(cond3_indexes,:));
spmi_two = spm_two.inference(p_critical, 'two_tailed', true, 'interp',true);
spmi_two.plot();

% Condition 2 vs Condition 3
figure()
spm_three = spm1d.stats.ttest_paired(ang(cond2_indexes,:), ang(cond3_indexes,:));
spmi_three = spm_three.inference(p_critical, 'two_tailed', true, 'interp',true);
spmi_three.plot();

[0todd0000]

Everything looks fine in the script. I have two follow-up questions:

Can you please confirm that the index vectors are defined as follows?

cond1_indexes = A==1;
cond2_indexes = A==2;
cond3_indexes = A==3;

Can you please confirm that the following three vectors are identical?

subj1 = SUBJ(cond1_indexes);
subj2 = SUBJ(cond2_indexes);
subj3 = SUBJ(cond3_indexes);

[joshuatbland]

Yes, I was able to confirm both of these.

[0todd0000]

OK, thanks. Can you please tell me:

• the number of subjects
• the shape of the Y array

[joshuatbland]

• Number of subjects = 6
• Shape of Y array = 6 by 828

[0todd0000]

OK, thank you for these details. There appears to have been a misorganization of Y.

Each row of Y should be a single 1D observation with Q points. Usually Q=100 or Q=101.

Since there are six subjects and three conditions, and assuming there is one observation per subject/condition pair, then the shape of Y should be: (18,Q). And A and SUBJ should look something like this:

A    = [1, 1, 1, 1, 1, 1,    2, 2, 2, 2, 2, 2,    3, 3, 3, 3, 3, 3]';
SUBJ = [1, 2, 3, 4, 5, 6,    1, 2, 3, 4, 5, 6,    1, 2, 3, 4, 5, 6]';

[joshuatbland]

Oh, sorry - I was looking at the shape of the wrong variable. The shape of the Y array is actually 18 by 828 (Q of 828).

[joshuatbland]

Is it okay to have Q be 828 as long as all of the observations have 828 points?

[0todd0000]

Q=828 is no problem.

However, I am confused: why do each of the individual subject figures in your original post show Q closer to 100?

Also, can you please send the values of A and SUBJ and/or verify that your A and SUBJ values are as follows?

A    = [1, 1, 1, 1, 1, 1,    2, 2, 2, 2, 2, 2,    3, 3, 3, 3, 3, 3]';
SUBJ = [1, 2, 3, 4, 5, 6,    1, 2, 3, 4, 5, 6,    1, 2, 3, 4, 5, 6]';