mutivariate anova tests

[joe198989]

Hi Tod,

Thanks for the great toolbox. I am a novice user and I'd like to know more about some of the statistical tests that I can use in my research.

The aim of my research is to compare the EMG of the normal subjects with CP patients. I have three non-directed null hypotheses. 1: There is no significant difference in normal subjects EMG during gait at different speeds (slow, moderate, fast) 2: There is no significant difference in CP subjects EMG during gait at different speeds (slow, moderate, fast) 3: There is no significant difference between Normal and CP subjects at each speed level?

I understand SPM package lacks multivariate tests which can do ANOVA with 2 factors (taking into account all three null hypotheses as one).

Q1: Is it justifiable to have three hypotheses rather than one and perform MANOVA (for the 1st and 2nd hypothesis) and Hotelling t- test (for the 3rd hypothesis). It is worth mentioning, I designed my hypotheses prior to conducting the experiment.

Q2: would I have gotten the same results had I performed multivariate ANOVA with 2 factors?How the results would differ if one hypothesis were used instead of three hypotheses?

Q3: Since MANOVA rm has not yet been developed in SPM could I justify using MANOVA for the 1st and 2nd hypotheses and using Hotelling t-test as multivariate post-hoc test even though the same subject performed the activity across speeds? If not what needs to be done to consider EMG covariance and time series in this scenario?

Thanks!

[0todd0000]

Hi, thank you for your question, and apologies for the limited MANOVA functionality.

Q1: The three null hypotheses are implicit in a two-way design, so it's fine. Since there is no specific prediction made, the study can be regarded as an exploratory one which searches for effects of speed and group (Normal vs. CP) on EMG. However, if each subject performed each of the three speeds then it is not suitable to use spm1d.stats.manova1 (even in post hoc analysis), because this function implements between-subjects analysis. You'd need within-subjects (i.e. repeated measures) analysis. For this study I'd recommend trying (a) paired Hotelling's T2 tests to compare the different speeds (six tests), and (b) two-sample Hotelling's T2 tests to compare the groups at each speed (three tests). Using a Bonferroni correction across these nine tests would be appropriate, albeit conservative.

Q2: Two-way MANOVA should yield results similar to those that are produced by the Hotelling's T2 tests mentioned above. The results will not be exactly the same, but they will be close. The main reason they will not be the same is that multivariate tests use fancier corrections than the Bonferroni correction. Since this is an exploratory study I think it's fine to use the more conservative Bonferroni correction.

Q3: As mentioned above, spm1d.stats.manova1 should only be used if the slow speed subjects were different from the moderate speed subjects, who were in turn different from the fast speed subjects. Hotelling's T2 tests adequately consider covariance amongst the EMG channels. The only real disadvantage of skipping to post hoc tests (e.g. Hotelling's T2 tests) is that you can not report a single p value for SPEED and GROUP (nor their interaction). Nevertheless, the magnitude of these effects will be implicit in the Hotelling's T2 tests.

Todd

[joe198989]

thanks for your quick response!

I am not sure what you meant by Bonferroni correction in response to Q1 hypothesis 1: paired Hotelling's T2 tests (6 tests) hypothesis 2: paired Hotelling's T2 tests (6 tests) hypothesis 3: Hotelling T2 tests (3 tests) Q1:Do you mean instead of using alpha level = 0.05 for the 1st, 2nd and 3rd hypotheses I need to use significant level of 0.05/9 = 0.005??

On another note, I am also interested in individual muscle activation comparison for the application of FES and assistive robotic control using individual muscle. I understand in this scenario muscle covariance is neglected but since the focus of this study is on certain muscles; Q2: Is it correct to do the following for individual muscle comparison Hypothesis 1: main analysis: ANOVARM (alpha =0.05) and post-hoc (paired t-test with alpha 0.016 ) Hypothesis 2: main analysis: ANOVARM (alpha =0.05) and post-hoc (paired t-test with alpha 0.016 ) Hypothesis 3: main analysis: t-test (alpha =0.05) and post-hoc

This certainly can be done using ANOVA2onerm but again the hypotheses were designed before the experiment was carried out and from your response to the above questions, I believe the results should not be different in this scenario as well.

Sorry, the next question is for me to understand the theory behind ANOVArm within and between subject interactions. Q3: I ve seen in the example provided in SPM package (anova1rm) the between-subject interaction was also considered. I am not entirely sure how important that would be in my study but would you please shed some light on the discrepancy that may occur in the results of within and between subjects (could the significant regions differ from each other and why this may happen?

[0todd0000]

Q1. A Bonferroni correction for N tests can be calculated with:

alpha = 0.05;
N = 9;
p_crit = spm1d.util.p_critical_bonf(alpha, N);

It is roughly the same as alpha/N. Then use p_crit instead of alpha when conducting inference. For example:

spm       = spm1d.stats.hotellings_paired(YA, YB);
spmi      = spm.inference(p_crit);

The Bonferroni correction is accurate when the $N$ tests are independent. For post hoc analysis it is usually slightly conservative.

Q2. That sounds fine, but I'd add the following:

• The family-wise error (alpha) is still 0.05; only critical p values change.
• The correction has to pertain to the original experimental design, so 0.016 is unsuitable. This correction would only be suitable if you had measured just one muscle. If you want to conduct RM-ANOVA (or paired t tests) on single muscles, then you need to calculate how many different RM-ANOVA tests (or paired t tests) can be conducted using the data you collected, then use this number as N in the Bonferroni correction. In other words, measuring lots of muscles, in lots of different conditions, makes an experimental less able to detect specific effects of specific muscles. As above, the Bonferroni correction may be too conservative, but is unfortunately the only post hoc procedure currently supported in spm1d.
• There are no new hypotheses at the post hoc stage. The post hoc analyses are meant to qualify the original hypotheses.
• In exploratory studies it doesn't really matter what is done for post hoc anlaysis. It is generally more important to verify exploratory findings using an independent datasets than it is to adopt correct post hoc procedures in the exploratory analysis. In other words, exploratory analyses search a dataset for possible effects. Confirmatory analyses test specific predictions. Hypothesis testing is more directly related to confirmatory analyses. It can also be used as a tool for exploratory analysis, but adopting correct hypothesis testing procedures does not change an exploratory study into a confirmatory one.

Q3. Whenever there are two experimental factors (in this case: SPEED and GROUP) they can interact, and when an interaction exists the main effects are generally not useful to interpret. In this case a significant interaction effect would indicate that SPEED induces different EMG changes for different groups. Without specific results it is difficult to discuss interpretations of regional effects, so I'd recommend general resources online like this one: http://statisticsbyjim.com/regression/interaction-effects/ The SPM interpretation of interactions is no different. The only exception is that the interaction can change across the measurement domain (i.e. between 0% and 100% time)/.

[joe198989]

Thanks for the prompt response!

Regarding Q1; after performing the main analysis using a Bonferroni correction (N=9) I found significant differences in the pair-hotelling t-test results between some of the speeds and hotelling t-test between population for each speed.

*Q1A In normal subjects significant difference was only found in paired Hotelling T2 between Normal-Slow. Since Hypothesis 1 stated: there is no significant difference in normal subjects EMG during gait at different speeds (slow, moderate, fast). Should I just make a general comment that there is a significant difference between EMG of normal subjects across speeds or it needs to be specified between Normal-Slow speed only???**

I want to explore the results further using the paired t-test for the hypothesis 1 and 2 and two-sample t-test for the hypothesis 3 in order to assess which of the vector components are significantly different. Since there is no multivariate post-hoc analysis in SPM and univariate post-hoc analysis is the only option available: can I use the following code for the univariate post-hoc test?

Note: I have 4 components (i.e., EMG) which I want to test.

Q1B**) I am not sure about the values for alpha and nTests to be used in the codes ?

%% post hoc for Hypothesis 1

**_alpha = ???
nTests = ???_**
p_critical_sidak = spm1d.util.p_critical_bonf(alpha, nTests);
spm       = spm1d.stats.ttest_paired(Control_slow(:,:,1), Control_Normal(:,:,1));
spmi      = spm.inference(p_critical_sidak, 'two_tailed',true, 'interp',true);
disp(spmi)

%% post hoc for Hypothesis 2

**_alpha = ???
nTests = ???_**
p_critical_sidak = spm1d.util.p_critical_bonf(alpha, nTests);
spm       = spm1d.stats.ttest_paired(CP_Slow(:,:,1), CP_Normal(:,:,1));
spmi      = spm.inference(p_critical_sidak, 'two_tailed',true, 'interp',true);
disp(spmi)

%% post hoc for Hypothesis 3

**_alpha = ???
nTests = ???_**
p_critical_sidak = spm1d.util.p_critical_bonf(alpha, nTests);
spm       = spm1d.stats.ttest2(Control_Slow(:,:,1), CP_Slow(:,:,1));
spmi      = spm.inference(p_critical_sidak, 'two_tailed',true, 'interp',true);
disp(spmi)

Regarding Q2 in the previous post I am not sure if alpha = 0.05 should be used: I have used the following for each individual muscle Note: I am testing 4 muscles in total

Q2A** Are the following correct or the alpha needs to be changed?? %% Main analysis Hypothesis 1

spm       = spm1d.stats.anova1rm(RF_Control, Speed_Control, Subj_Control)
spmi      = spm.inference(0.05);
disp(spmi)

%% Main analysis Hyptohesis 2

%(1) Conduct SPM analysis:
spm       = spm1d.stats.anova1rm(RF_CP,Speed_Control, Subj_Control)
spmi      = spm.inference(0.05);
disp(spmi)

%% Main analysis Hypothesis 3

 %(1) Conduct SPM analysis:
spm       = spm1d.stats.ttest2(RF_Control, RF_CP);
spmi      = spm.inference(0.05, 'two_tailed',true, 'interp',true);
disp(spmi)

Thanks!

[0todd0000]

Q1. The stated hypothesis is slightly incorrect. Since this is a two-way MANOVA design, the null hypotheses are: (1) no main effect of SPEED, (2) no main effect of GROUP, and (3) no interaction effect on the four-muscle EMG trajectories. The result you mention suggests that (1) is rejected. The hypothesis: "there is no significant difference in normal subjects EMG during gait at different speeds" is not consistent with the experimental design. If this truly were the hypothesis, you would have measured only normal subjects. All results should be interpreted in the context of the original design. So if analyses reach significance please first relate those results to null hypotheses (1), (2) and/or (3) above, then secondarily mention which group(s) and/or which muscle(s) were likely responsible for the (1), (2) and/or (3) results.

Q1B. Alpha is the a priori Type I error rate and pertains to the entire experiment. It is conventionally 0.05, so 0.05 should be used unless you have justification to use a different value. The number of total tests is tougher. Strictly speaking, it is (IJM - choose - 2) = (IJM) ! / ( (IJM -2)! 2! ), where I, J and M are the numbers of FACTOR-1 conditions, FACTOR-2 conditions and dependent variables, respectively. In your case I=2, J=3, and M is 4, so there are 276 different two-mean tests possible. However, please note that this issue is rather complex. Please see the link below, which discusses the complexities of post hoc comparisons in two-way ANOVA: https://www.mathstat.dal.ca/~stat2080/Fall14/Lecturenotes/mult2way1.pdf If you search the internet for "post hoc comparisons in two-way ANOVA" you will find a number of discussions that emphasize how complex this issue is. Since this experiment's design is even more complex than that (i.e. MANOVA), the post hoc procedures are also even more complex. I think the only general approach is to regard post hoc tests as qualifying the original hypotheses, and you can do anything you want in post hoc tests, provided they do not disagree with higher-level analyses. That is, single-muscle t test results should not disagree with multiple-muscle T2 test results, which in turn should not disagree with MANOVA results, and single-muscle t test results should also not disagree with single-muscle ANOVA results. If disagreement exists in lower-level analyses then they have been conducted incorrectly and/or an inappropriate threshold has been used. Also please note that any p values computed following adjusted p thresholds need to be back-corrected as described here (see bottom of the page): http://www.spm1d.org/doc/PostHoc/anova.html

Q2A. One approach to calculating a corrected threshold for anova1rm or for ttest2 is to calculate how many of each type of test is possible, then use a Bonferroni correction. However, as mentioned above, the issue is complex and it is not generally possible to describe a single correct solution.

[joe198989]

Hi Tod,

I have a couple of questions about co-contraction and muscle synergism.

%%%%%%%%% Q1 %%%%%%%%% I'd like to compare the EMG from able-bodied and CP patients in terms of ankle co-contraction and knee co-contraction. Therefore, in case of ankle muscle co-contraction, I combined the EMG data from the Triceps surae (i.e., Gast med, Gast lat and Soleus) and tibialis anterior to perform Hotelling T2.

my question is about the inter-muscle covariance which in this case is only between the triceps surae muscles, and tibialis anterior is not temporally correlated to the triceps surae. In addition, I have seen in one of your publications (Robinson et al (2015)-Statistical Parametric Mapping (SPM) for alpha-based statistical analyses of multimuscle EMG time-series) that agonist and antagonist muscles are used in the analysis. 1A) Wouldn't that contradict the concept behind using Hotelling's T2?

%%%%%%%%% Q2 %%%%%%%%% Muscle synergy analysis is performed to separate the time-varying component known as excitation primitive and spatially fixed synergy vectors. Each one of these excitation primitives is input to each synergy vector which has its own temporal characteristics.

2) Although these excitation primitives are not temporally correlated could I assume they are part of the CPG and combine them in order to perform Hotelling T2 between the able-bodied and CP patients

3) Mainly I'd like to understand even if the signals are not temporally correlated would it be alright to use Hotelling T2 as long as my hypothesis pertains to all the signals included in the experiment?

[0todd0000]

1) This doesn't contradict Hotelling's test concepts, because this test and all other multivariate procedures quantify the degree of (in)dependence amongst multiple dependent variables. The variables could be completely dependent (i.e. effectively one variable), completely independent (i.e. effectively two variables), or somewhere in between.

2) I'm not sure what you mean by "combine". Hotelling's tests pertains to mean vectors, and it's fine to use arbitrary variables as components of those vectors provided the mean vectors represent your null hypothesis.

3) Yes. Perfectly uncorrelated variables is just a special case of multivariate procedures. If they are indeed uncorrelated results should be the same as analyzing the variables separately using a Bonferroni correction, which assumes variable independence.

[joe198989]

Sorry for the follow-up question since I am a novice user I am having difficulty to comprehend the differences between the Hotelling's T2 and two-sample t-test. From my understanding the Hotelling's T2 considers the inter-component covariance whereas t-test does not. 1) Is there any other difference between these two tests?****

So in case of muscle synergism, 4 excitation primitives (not correlated) are populated from the EMG signals of each group. When I say combine them I mean put the mean vector of each excitation primitive in one matrix which essentially comprised of 4-component vector-field I = 4 (# excitation primitives), J = 40 (total number of subjects abled bodied and CP), Q = 100. 2) Is this a right way to do it?

My hypothesis for considering these excitation primitives as one matrix is that at the spinal cord level these 4 primitives are combined together but am not sure if this has anything to do with the inter-component covariance and if Hotelling T2 is appropriate for this hypothesis? 3) Can I use Hotelling's T2 for the aforementioned hypothesis?**

I'd really appreciate your help!

[ghost]

1) If there are I separate variables, and all I variables are mutually independent, then a Hotelling's T2 test is equivalent to I separate t tests with a Bonferroni correction. If the I variables are perfectly correlated then a Hotelling's T2 test is equivalent to a single t test. Usually reality is somewhere between these two extremes. Please find additional useful details here: http://www.real-statistics.com/multivariate-statistics/hotellings-t-square-statistic/hotellings-t-square-independent-samples/

2) If your null hypothesis pertains to mean vector fields (i.e. 100x4 arrays), then this sounds correct.

3) It's tough to say because "combined together" is not very precise. A null hypothesis must be a precise assertion of equivalence. "Combined together", by itself, is not an assertion of equivalence so is not a valid null hypothesis. The default null hypothesis for a two-sample Hotelling's T2 test is that the two mean vector fields are equivalent. In your case this would be H0: m1 = m2, where m1 and m2 represent two (100 x 4) mean vector fields from two different populations of subjects. I don't think this pertains to a hypothesis regarding the combination of multiple variables. It sounds like you might be interested in quantifying how well a linear combination of primitives can explain the data. That type of analysis is valid, but is not directly related to the Hotelling's T2 test, and is not supported in spm1d.

[joe198989]

Thank you for your explanation! 1) Hypothetically if significant difference occurred between two populations using Hotelling's T2 could one interpret that the linear combination of these excitation primitives reach the threshold (T2 statistics)

On another note, there are different features in the signal that can be considered to compare between-subjects namely A) magnitude and B) shape or pattern 2) Does SPM consider both of these features? 3) if a significant difference is observed can one conclude both magnitude and pattern of the signal are different between-populations?

[0todd0000]

Question 1: No, a linear combination is a scalar, like:

z = (2 * x1) + (1.5 * x2) + (0.8 * x3) + (2.5 * x4);

where x1 through x4 are four dependent variables.

In contrast, a two-sample Hotelling's T2 test assesses mean vectors, like:

z = [x1 x2 x3 x4];

Questions 2 & 3: These questions are very good, but they're common to all tests and not just to Hotelling's T2 test, so I've started a new discussion here: #75.