[Feature Request]: RM MANOVA

keslin527 commented 2 years ago

Description

repeated measures/mixed MANOVA

Purpose

To analyze related outcomes with >= 1 between subjects IV and >1 within subjects IV

Use-case

This would be extremely useful to analyze repeated measures >=1 (IV) with several outcomes, or (mixed) factorial multivariate research designs.

Is your feature request related to a problem?

not applicable, outside of not having the feature.

Describe the solution you would like

Would it be possible to create this? Likely under the repeated measures or MANOVA tab?

Describe alternatives that you have considered

SPSS has been the only one so far.

Additional context

Outside of the above, I can't offer much more information. Regardless, thank you for all you do to push this project forward. It has been a lifesaver for teaching and research.

tomtomme commented 7 months ago

@keslin527 @JohnnyDoorn Isn't this already possible with our RM ANOVA module? You can specify >1 DVs (repeated measures factors) that depend on time. You can also include >1 between subject factors and even >1 covariates. This makes it an RM (M)AN(C)OVA effectively. Maybe we should rename the module to reflect this? Or do I have a conceptual misunderstanding here?

JohnnyDoorn commented 7 months ago

@keslin527 could you maybe provide a concrete data example for this use-case? I think that will clarify to us what is still needed on the JASP side of things to accommodate.

keslin527 commented 7 months ago

Thank you for responding! I guess I am wondering; should there be a box that specifies the exact dependent variables to be measured? Would adding all the levels of the variables in the Repeated Measures Factors box tell the software that all the DV's are basically the same DV under different contexts? Or is there another way to do it in JASP?

On Tue, Feb 20, 2024 at 2:44 AM Thomas Langkamp @.***> wrote:

@keslin527 https://github.com/keslin527 @JohnnyDoorn https://github.com/JohnnyDoorn Isn't this already possible with our RM ANOVA module? You can specify >1 DVs (repeated measures factors) that depend on time. You can also include >1 between subject factors and even >1 covariates. This makes it an RM (M)AN(C)OVA effectively. Maybe we should rename the module to reflect this? Or do I have a conceptual misunderstanding here?

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/1660#issuecomment-1953830539, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYHWESLVGQGU2DRNC6EQIATYURV6PAVCNFSM5QZ3HAN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJVGM4DGMBVGM4Q . You are receiving this because you were mentioned.Message ID: @.***>

keslin527 commented 7 months ago

Thank you for responding! I guess I am wondering; should there be a box that specifies the exact dependent variables to be measured? Would adding all the levels of the variables in the Repeated Measures Factors box tell the software that all the DV's are basically the same DV under different contexts? Or is there another way to do it in JASP?

On Tue, Feb 20, 2024 at 2:57 AM Johnny van Doorn @.***> wrote:

@keslin527 https://github.com/keslin527 could you maybe provide a concrete data example for this use-case? I think that will clarify to us what is still needed on the JASP side of things to accommodate.

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/1660#issuecomment-1953855218, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYHWESNQSS5NZSOBQ7ZIMMLYURXSPAVCNFSM5QZ3HAN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJVGM4DKNJSGE4A . You are receiving this because you were mentioned.Message ID: @.***>

keslin527 commented 7 months ago

Let's say I wanted to look at pre-post systolic and diastolic blood pressure in the "repeated measures factors" box, how do I tell the software to differentiate between systolic as one variable measured in 2 contexts and diastolic as another without making the software think I'm measuring the same variable 4 times? In SPSS it looks like the image attached below:

[image: RM_Manova_image.PNG]

On Tue, Feb 20, 2024 at 2:57 AM Johnny van Doorn @.***> wrote:

@keslin527 https://github.com/keslin527 could you maybe provide a concrete data example for this use-case? I think that will clarify to us what is still needed on the JASP side of things to accommodate.

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/1660#issuecomment-1953855218, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYHWESNQSS5NZSOBQ7ZIMMLYURXSPAVCNFSM5QZ3HAN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJVGM4DKNJSGE4A . You are receiving this because you were mentioned.Message ID: @.***>

tomtomme commented 7 months ago

@keslin527 Thx for the details. Please upload the image via the github webpage again. Uploading via eMail does not work.

JohnnyDoorn commented 7 months ago

Hi @keslin527 , If I understand correctly, you want to add two RM factors; there is a gif illustration of how to do this (where to click) here: https://johnnydoorn.github.io/statistics-lectures/courses/SSR/SSR_2023-2024/MiscFiles_J/JASP_Gifs/RM-ANOVA_SpecifyFactors.gif Where two RM factors are specified (Alcohol & speed, each with 3 levels). JASP will then see these two as separate variables, and since they are both measured within subject, will need you to specify 3x3 cells in the input box below. Does this clarify the analysis?

Kind regards Johnny

keslin527 commented 7 months ago

RM_ANOVA_image

Thanks for replying so quickly. Can you all see the image now? Johnny-your graphic appears to be two within IV's (independent variables; speed & alcohol consumption) and 1 between IV (ambient light, as expressed by "daytime") on some outcome (what is it, by the way?). So the Repeated Measures Factors box is great for adding the two within IV's.

What I am asking is, what if there is another DV (outcome/dependent variable)? There would be 9 more columns in the dataset for each of the levels for speed and alcohol. It appears that adding these extra 9 rows into the Repeated Measures Factors box would just be read as 9 more contexts of the variable, rather than as a separate variable. Is there a way in this module to distinguish between two separate outcomes?

tomtomme commented 7 months ago

@keslin527 Yes, johnny named the 2 RM facors like IVs. Both RM Factors are splittet in 3 time steps. And both could measure different DVs at those times - the animation just does not show the names of the DV(s). So Johnny also could just have named those two factors like the DVs that are measured at the 3 time steps. So this makes no difference. If you put >1 RM factors in, you can also accomodate >1 DV at the same time.

You could for example measure the two DVs systolic and diastolic from your picture. And you could measure it for Johnys speed and alcohol consumption IVs.

Assuming that the data is in wide format you would then need the following columns to accomadate 4 Factors with the 2 IVs and the 2 DVs:

RM Factor 1

speed_systolic at time 1 speed_systolic at time 2 speed_systolic at time 3

RM Factor 2

speed_diastolic at time 1 speed_diastolic at time 2 speed_diastolic at time 3

RM Factor 3

alc_systolic at time 1 alc_systolic at time 2 alc_systolic at time 3

RM Factor 4

alc_diastolic at time 1 alc_diastolic at time 2 alc_diastolic at time 3

So this results in 4 RM Factors. Put all 4 in the RM ANOVA module and you essentially get a RM MANOVA over 2 within IVs and DVs. Right?

But maybe you are thinking in a "long format" data structure instead of "wide format" one. Then the data layout would look different then described above. At the moment however JASPs RM ANOVA module needs the data to come in wide format.

P.S. I hope my above solution is possible with current implementation of jasp. I could not find however a dataset in jasps data library which does this.

keslin527 commented 6 months ago

I apologize for not explaining myself more clearly. Thank you for your patience!

The way Johnny has his setup appears to be a 2-way (3x3) ANOVA. Obviously, I don’t have his dataset, so I’ll try to explain in more detail using mine. For my research question, I’d like to look at the effect of a pre/post intervention of systolic and diastolic blood pressure. First I’ll demonstrate your proposed analysis with the output on SPSS and JASP. I’ll then demonstrate the JASP analysis I am asking for, on SPSS. In SPSS, the way you propose looks like this with one factor (time) and one outcome (blood pressure).

As demonstrated below, the data are in wide format with pre- and post-systolic pressures pasted in the “Within-Subjects Variables" box below. This is analogous to the "Repeated Measures Cells" in JASP. We can ask for post hoc tests, effect sizes, etc. which I will not post on here.

After we do this, we get the output below with pre-and post-blood pressures as each of 4 contexts of a pressure variable. However, what if I wanted to look at other variables like cholesterol, this would not work because the software is looking at each variable as a context rather than as a separate variable! Additionally, SPSS will offer a Multivariate Tests table which appears to be a sort of profile analysis. Does JASP currently offer this feature for the Repeated Measures ANOVA suite?

SPSS then offers Univariate tests under Tests of within-Subjects Effects/Contrasts as shown below.

These are the same outputs offered by JASP as shown in the pictures below. Again, this is not appropriate because we have different variables being compared as if they are just 4 different contexts of 1 variable, when we are trying to look at it as 2 contexts (pre + post) of 2 separate variables (systolic BP and diastolic BP). What if we also had cholesterol, blood sugar etc.? These are all different variables that should not be compared together in a post hoc test. Therefore, this manner of analysis is improper.

JASP setup:

JASP Output:

The feature I am requesting is below, as demonstrated on SPSS. Again, I’m looking for the multivariate result for two levels on time for two separate outcomes (systolic and diastolic). Potentially I may want to add k more outcomes to the analysis (cholesterol, blood sugar, etc). In SPSS these would be added to the Measure Name box, and each variable is labeled separately as shown below. The Measure Name box is where separate dependent variables should be added in order to let the software know we’re analyzing these as separate outcomes. Is this already possible in JASP? If not, would it be possible to add it?

Once we do this, then we can add the two levels for systolic pressure, and diastolic pressure as shown below. Again, note these data are in wide format.

The outputs are shown below. It lists the two DV’s and offers descriptives for each. It also offers a multivariate test output with a p-value and partial eta squared for the multivariate output. Mauchly’s test of sphericity then evaluates the two (or more) outcome variables separately. You will note SPSS will repeat the multivariate output several times.

Following the multivariate output, it offers univariate output separate for each outcome as shown below.

Finally, post hoc tests for these univariate ANOVA’s are offered as shown in the Pairwise Comparisons box below. The post hoc tests evaluates each variable separately. This is more preferable than the first example where variables measured differently are all in the same post hoc test.

What I am asking is can you implement the following features into JASP?

Adding a Measure Name (i.e., dependent variable) box where the user can differentiate between the k different dependent variables. Adding them to the Repeated Measures Factors box in JASP is not appropriate, because the software just sees it as a RM ANOVA with different contexts instead of different outcomes.
A multivariate test for the Repeated Measures MANOVA?

OR

If it already does this, would you be able to explain how?

Thank you once again for your time and patience.

tomtomme commented 6 months ago

I tried to explain this above. Jhonny too, I think. From my perspective JASPs Repeated Measures Factors box is appropriate. Why do you think that JASP treats different Factors as only one DV? But maybe you have not recognized this? This is where you would initialize and name the second DV. Also the post hoc tests then show separate tables for those DVs (and tables for interactions) Here is an example file with such an analysis from jasps data library: AlcoholAttitudes.jasp.zip Remove the .zip before opening.
This one is a bit more tricky in JASP. You would need to convert your data from wide to long. We are working on GUI elements for the data edit mode, to make this easy. But I can give no timeline for when this will be done. So at the moment you would need to convert from wide to long e.g. in excel. After that you could use the MANOVA module to get Pillays Trace etc. The problem with this solution: I do not know if MANOVA and RM MANOVA rely on the exact same assumptions. Would be cool if you test this with your data and give me feedback, if this gives you the same results as in SPSS.

JohnnyDoorn commented 6 months ago

To add to @tomtomme , we do not provide these multivariate tests (MANOVA is not very high on our priority list unfortunately), but I think the most important tests for these types of research scenario's are captured by RM ANOVA. Here is a video to create the structure that (I think) you are looking for:

https://github.com/jasp-stats/jasp-issues/assets/15704203/833447f8-a150-4dd5-856f-d6ac31927dda

keslin527 commented 6 months ago

I haven't had a chance to use the alcohol dataset as yet, but I can say off the bat that MANOVA and RM MANOVA do not share the same assumptions. Asking for Pillai's Trace through the MANOVA route would be incorrect, as using regular MANOVA with repeated measures data violates the assumption of the independence of observations (which is why RM MANOVA is necessary). This inflates Type I error.

I did run the blood pressure dataset the way Johnny suggested, and the outputs still differ from that of SPSS; I am not sure that adding the outcomes as a factor is the correct way to do it.

That said, I very much appreciate the honest response that MANOVA is not a priority for the team at this time; JASP is free and I suppose that beggars cannot be choosers. That said, if it ever does become a priority once again, it would be very helpful for intervention studies in health sciences studies as RM MANOVA is a powerful tool in these areas. I've also taken the liberty to attach a paper for that purpose; I hope that you do not mind.

Thank you so much again for your help and patience. Have a great week!

MANOVA Method for Analyzing Repeated Measures Designs_An Extensive Primer.pdf

tomtomme commented 6 months ago

A paper is always helpful. Also all the details you added to the conversation. So thank you and lets hope that someone comes along to implement this!

keslin527 commented 6 months ago

No problem! Here's another link in case this helps anyone reading this thread who may be able to help:

https://statistics.laerd.com/spss-tutorials/one-way-repeated-measures-manova-using-spss-statistics.php

tomtomme commented 2 weeks ago

Summing up everything about RM MANOVA so far:

Rationale:

It might be possible to do RM MANOVA, split plot and nested designs in JASP via GLM or GLMM option, but nobody from the team nor the OP of #2892 are sure how.
RM MANOVA currently in JASP can be done partly via RM ANOVA, e.g. Pillais Trace is missing. Getting this from normal MANOVA violates assumptions of independence. Also adding the DVs to the Repeated Measures Factors box in JASP is not appropriate, because the software just sees it as a RM ANOVA with different contexts instead of different outcomes.
we need RM MANOVA for agricultural designs like split-plot and health sciences (following examples from above and #793 #1682 #2892

Examples:

pre-post systolic and diastolic blood pressure to differentiate between systolic as one variable measured in 2 contexts and diastolic as another without making the software think I'm measuring the same variable 4 times. Detailed in SPSS with screenshots: https://github.com/jasp-stats/jasp-issues/issues/1660#issuecomment-1967412560
Agricultural Example (Split Plot or SPlot): study the effects of 3 Nitrogen level and 2 irrigation regimes on the growth parameters and yield of maize plants in pots arranged as CRD. For each factor combinations, the samples were replicated 5 times. Every week, the height, leaf numbers and leave SPAD readings were taken.IV - Nitrogen level and Water regimeDV - Height, Leave numbers, SPAD readings (measured every week until harvest)

Thus we need a new RM MANOVA module or more features within the RM ANOVA module like:

A Measure Name (DV) box where the user can differentiate between the k different DVs.
A multivariate test for the Repeated Measures MANOVA (Pillai's Trace)

Material:

R package: multRM() function
Bathke, A. C., Friedrich, S., Pauly, M., Konietschke, F., Staffen, W., Strobl, N., & Höller, Y. (2018). Testing Mean Differences among Groups: Multivariate and Repeated Measures Analysis with Minimal Assumptions. Multivariate Behavioral Research, 53(3), 348. https://doi.org/10.1080/00273171.2018.1446320
Friedrich, S., & Pauly, M. (2018). MATS: Inference for potentially singular and heteroscedastic MANOVA. Journal of Multivariate Analysis, 165, 166–179. https://doi.org/10.1016/j.jmva.2017.12.008
Friedrich, S., Konietschke, F., & Pauly, M. (2019). Resampling-Based Analysis of Multivariate Data and Repeated Measures Designs with the R Package MANOVA.RM. The R Journal, 11(2), 380. https://doi.org/10.32614/RJ-2019-051
O’Brien, R. G., & Kaiser, M. K. (1985). MANOVA method for analyzing repeated measures designs: An extensive primer. Psychological Bulletin, 97(2), 316–333. https://doi.org/10.1037/0033-2909.97.2.316
MANOVA Method for Analyzing Repeated Measures Designs_An Extensive Primer.pdf
https://statistics.laerd.com/spss-tutorials/one-way-repeated-measures-manova-using-spss-statistics.php

keslin527 commented 2 weeks ago

I appreciate the update; I was unaware (though not surprised) that others requested the feature. The feature's use in the field of agriculture is also duly noted.

Thank you.

jasp-stats / jasp-issues