Kinematic, kinetic and EMG data

[gabrielmoisan]

Hi Todd, I am having trouble with a reviewer, clearly not a fan of SPM in the analysis of biomechanical data. I don't know how to answer his concerns in a satisfactory way. He is insistent and not satisfied by my answers. Can you help me? I used non-parametric permutations (Nichols and Holmes) to compare joint angles and moments and EMG (6 muscles) during shod and barefoot walking (stance phase only).

Reviewer's comment: An issue related to this statistical analysis is permutation inference expects independence assumption, but it can be relaxed if the joint probability does not change with permutation. Data in the time series provided by joint angles and moments, and muscle activation is not independent, therefore it is not clear why this analysis should be applied to this data. Then, in my opinion, if I can not believe the data analysis is correct, I can not endorse the discussion based on it. I need the authors to better explain their approach, and mostly show me how to overcome my concerns.

He also suggests to use ANCOVA with walking speed as a covariant. Am I wrong by saying that ANCOVA has not been validated yet in SPM?

Thanks a lot for you time, Gabriel Moisan PhD Université du Québec à Trois-Rivières, Canada

[gabrielmoisan]

Hi again, just to be sure, what are the statistical tests behind these two: -spm1d.stats.nonparam.ttest_paired(yA, yB)

• spm1d.stats.nonparam.ttest2(yA, yB) Thanks, Gabriel

[0todd0000]

Hi Gabriel,

This is a very tough question! There are many aspects to consider, and it will be difficult to address them all. I will address the main issue of the reviewer's comment here, because it directly pertains to the title of this issue: "Kinematic, kinetic and EMG data".

If you have other questions about spm1d (e.g. permutation, ANCOVA), please ask them as separate issues. (We are trying to keep this forum as modular as possible, so that other users can find answers efficiently. It's better to submit N questions as N separate issues than to include N questions in one issue.)

Regarding the main reviewer concern: I don't think the reviewer is criticizing SPM (at least in the comment you sent). I think instead the reviewer's comment pertains to a specific issue that is largely independent of SPM:

• Multivariate analysis and joint probabilities

The reviewer's comment suggests that many dependent variables (DVs) were analyzed, and that separate analyses were conducted for each DV. This is a problem, not only for SPM, but for any analyses; it is generally invalid to collect and separately analyze many DVs because many of those DVs are correlated (e.g. EMG levels and external forces are generally positively correlated). This correlation implies that the joint probability (between multiple DVs) must be considered when making inferences about multiple-DV effects.

The reviewer's comment implies that separate analysis of DVs (using t tests) is not valid in general, and I'd tend to agree. It would be better to conduct multivariate analyses, and not to use t tests.

However, one rebuttal you could make is that separate analyses with a Bonferroni correction for N tests (across the N DVs) is more conservative than multivariate techniques, so any discovered effects (after a Bonferroni correction) would certainly appear in a multivariate analysis. This rebuttal assumes that you have applied a Bonferroni (or other) correction across your N tests.

If you wish to follow-up about the multivariate DV issue, please do so below. If you wish to discuss other aspects of spm1d or SPM, including ANCOVA, please create new issues.

Todd

[gabrielmoisan]

Hi Todd, Thank you for the answer. I am a little confused to be honest. When I look at the biomechanics literature (walking studies), I don't see a lot of multivariate analyses. Are we all doing wrong analyses? The objective of my project is to compare shod vs barefoot walking at comfortable and fast speeds. Basically, 16 t-tests at confortable walking speed and 16 t-tests at fast walking speed (shod vs barefoot) which is the way most locomotion researchers does it. I did not use a Bonferroni correction as it would be very severe. Instead, I discuss this limitation in the interpretation of the results in the discussion section of the manuscript. Am I wrong in doing these analyses? Is there a way to convince the reviewer that it is a legitimate way to analyze our data?

If you suggest multivariate analyses, can you point towards the right direction (first time with multivariate here!). What kind of analyses do you recommend to implement in SPM?

Thanks a lot! Coming from a clinical background, you are of great help!

[0todd0000]

When I look at the biomechanics literature (walking studies), I don't see a lot of multivariate analyses.

I agree. In my view, this is a major problem in the literature.

Are we all doing wrong analyses?

"Wrong" is a strong word and also somewhat misleading. In reality the quality of analyses is much more nuanced, for a variety of reasons. For example:

• If observed effects are large, they will be large regardless of the analysis approach used.
• If the experiment is designed poorly, even superb analyses will produce questionable results.
• Multivariate data generally require multivariate analyses, but this does not imply that univariate analyses produce incorrect results.

I did not use a Bonferroni correction as it would be very severe. Instead, I discuss this limitation in the interpretation of the results in the discussion section of the manuscript. Am I wrong in doing these analyses? Is there a way to convince the reviewer that it is a legitimate way to analyze our data?

• No, this is not wrong. Again, "wrong" is too strong.
• This type of analysis is consistent with an exploratory perspective on experimentation, which is a perfectly valid form of experimentation.
• Tell reviewers that this was an exploratory design, with the main goal of detecting effects.
• I agree that a Bonferroni correction is probably too severe. However, not using a correction is also problematic. For example, if you measure and analyze 10 variables, you are approximately 40% likely to observe an effect that is not real (i.e., a false positive).

If you suggest multivariate analyses, can you point towards the right direction (first time with multivariate here!). What kind of analyses do you recommend to implement in SPM?

spm1d currently supports the following multivariate procedures:

• spm1d.stats.hotellings — one-sample Hotelling’s T2 test
• spm1d.stats.hotellings_paired — paired Hotelling’s T2 test
• spm1d.stats.hotellings2 — two-sample Hotelling’s T2 test
• spm1d.stats.cca — canonical correlation analysis (univariate 0D independent variable and multivariate 1D dependent variable)
• spm1d.stats.manova1 — one-way multivariate analysis of variance.

The first three are the multivariate versions of t tests.

[gabrielmoisan]

Hi Todd, Thanks so much for your time, I really appreciate it. For what I understand from your comment, I think I am better suited with MV analyses. I read your 2014 paper in Gait Posture and it really helped me understand.

I have follow-up questions in order to make sure to correctly implement the Hotelling's paired T2 in SPM. I have kinematics, kinetics and EMG variables (total of 16 variables) of 21 individuals during two experimental conditions (shod and barefoot) and two speeds (slow and fast). I don't want to compare to data at different walking speed (slow vs fast speed)

How many Hotelling's tests should I do? I see two possible options: Option 1: -Create a total of four 21 x 101 x 16 matrices (with all the kinematics/kinetics and EMG variables). 1= slow speed condition 1, 2= slow speed condition 2, 3= fast speed condition 1, 4=fast speed condition 2. -Implement two Hotelling's paired T2 in SPM (1 vs 2 and 3 vs 4)

Option 2 -Separate the kinematics (21 x 101 x 6), kinetics (21 x 101 x 6) and EMG (21 x 101 x 4) data in different matrices (total of 12 matrices) -Same comparisons as Option 1 but with 6 Hotelling's paired T2 instead of 2.

I hope I don't ask too much. I really appreciate your time. Gabriel

[0todd0000]

Option 1 will be difficult because the (co-)variance estimation will likely be a problem.

• For 2 dependent variables (DVs), there are three (co-)variance components: one variance component for each DV, and one covariance component describing the correlation between the two DVs. *For 3 DVs there are six components (3 variance + 3 covariance).
• For 16 DVs there are 136 components (16 variance + 120 covariance). A very large number of observations (thousands) would be needed to accurately estimate this covariance matrix.

So I think Option 2 is better, but I'd suggest another possibility: group by joint. Joint reaction forces, for example, have three components that describe the same physical property (i.e., a single three-dimensional force), so it make physical sense to group these three DVs (or actually, one multivariate DV). It makes less sense to group moments (or forces) across joints.

Note: SPM only solves one problem: DVs which vary over a continuous n-dimensional continuum (e.g. 1-dimensional time). It does not solve the inter-joint problem. As far as I know there is no statistical theory that can accurately model biomechanical systems involving multiple multivariate DVs at different joints and/or across different structures. So my only suggestion would be to try DV groupings in a way that makes most physical sense, then correct for multiple tests across the different DV groupings. Perhaps even try a variety of groupings to check whether the results are sensitive to particular groupings.

More generally, this is a matter of DV definition. If one does not clearly define the DV(s) of interest prior to the experiment, then it can become difficult to objectively choose appropriate DVs and/or analyses after the experiment has been conducted.

[gabrielmoisan]

Thank you so much Todd! I really appreciate your time and I learn a lot from you!