search

jasp-stats / jasp-issues

This repository is solely meant for reporting of bugs, feature requests and other issues in JASP.
59 stars 29 forks source link

[Feature Request]: RM MANOVA #1660

Open keslin527 opened 2 years ago

keslin527 commented 2 years ago

Description

repeated measures/mixed MANOVA

Purpose

To analyze related outcomes with >= 1 between subjects IV and >1 within subjects IV

Use-case

This would be extremely useful to analyze repeated measures >=1 (IV) with several outcomes, or (mixed) factorial multivariate research designs.

Is your feature request related to a problem?

not applicable, outside of not having the feature.

Describe the solution you would like

Would it be possible to create this? Likely under the repeated measures or MANOVA tab?

Describe alternatives that you have considered

SPSS has been the only one so far.

Additional context

Outside of the above, I can't offer much more information. Regardless, thank you for all you do to push this project forward. It has been a lifesaver for teaching and research.

tomtomme commented 9 months ago

@keslin527 @JohnnyDoorn Isn't this already possible with our RM ANOVA module? You can specify >1 DVs (repeated measures factors) that depend on time. You can also include >1 between subject factors and even >1 covariates. This makes it an RM (M)AN(C)OVA effectively. Maybe we should rename the module to reflect this? Or do I have a conceptual misunderstanding here?

JohnnyDoorn commented 9 months ago

@keslin527 could you maybe provide a concrete data example for this use-case? I think that will clarify to us what is still needed on the JASP side of things to accommodate.

keslin527 commented 9 months ago

Thank you for responding! I guess I am wondering; should there be a box that specifies the exact dependent variables to be measured? Would adding all the levels of the variables in the Repeated Measures Factors box tell the software that all the DV's are basically the same DV under different contexts? Or is there another way to do it in JASP?

On Tue, Feb 20, 2024 at 2:44 AM Thomas Langkamp @.***> wrote:

@keslin527 https://github.com/keslin527 @JohnnyDoorn https://github.com/JohnnyDoorn Isn't this already possible with our RM ANOVA module? You can specify >1 DVs (repeated measures factors) that depend on time. You can also include >1 between subject factors and even >1 covariates. This makes it an RM (M)AN(C)OVA effectively. Maybe we should rename the module to reflect this? Or do I have a conceptual misunderstanding here?

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/1660#issuecomment-1953830539, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYHWESLVGQGU2DRNC6EQIATYURV6PAVCNFSM5QZ3HAN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJVGM4DGMBVGM4Q . You are receiving this because you were mentioned.Message ID: @.***>

keslin527 commented 9 months ago

Thank you for responding! I guess I am wondering; should there be a box that specifies the exact dependent variables to be measured? Would adding all the levels of the variables in the Repeated Measures Factors box tell the software that all the DV's are basically the same DV under different contexts? Or is there another way to do it in JASP?

On Tue, Feb 20, 2024 at 2:57 AM Johnny van Doorn @.***> wrote:

@keslin527 https://github.com/keslin527 could you maybe provide a concrete data example for this use-case? I think that will clarify to us what is still needed on the JASP side of things to accommodate.

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/1660#issuecomment-1953855218, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYHWESNQSS5NZSOBQ7ZIMMLYURXSPAVCNFSM5QZ3HAN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJVGM4DKNJSGE4A . You are receiving this because you were mentioned.Message ID: @.***>

keslin527 commented 9 months ago

Let's say I wanted to look at pre-post systolic and diastolic blood pressure in the "repeated measures factors" box, how do I tell the software to differentiate between systolic as one variable measured in 2 contexts and diastolic as another without making the software think I'm measuring the same variable 4 times? In SPSS it looks like the image attached below:

[image: RM_Manova_image.PNG]

On Tue, Feb 20, 2024 at 2:57 AM Johnny van Doorn @.***> wrote:

@keslin527 https://github.com/keslin527 could you maybe provide a concrete data example for this use-case? I think that will clarify to us what is still needed on the JASP side of things to accommodate.

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/1660#issuecomment-1953855218, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYHWESNQSS5NZSOBQ7ZIMMLYURXSPAVCNFSM5QZ3HAN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJVGM4DKNJSGE4A . You are receiving this because you were mentioned.Message ID: @.***>

tomtomme commented 9 months ago

@keslin527 Thx for the details. Please upload the image via the github webpage again. Uploading via eMail does not work.

JohnnyDoorn commented 9 months ago

Hi @keslin527 , If I understand correctly, you want to add two RM factors; there is a gif illustration of how to do this (where to click) here: https://johnnydoorn.github.io/statistics-lectures/courses/SSR/SSR_2023-2024/MiscFiles_J/JASP_Gifs/RM-ANOVA_SpecifyFactors.gif Where two RM factors are specified (Alcohol & speed, each with 3 levels). JASP will then see these two as separate variables, and since they are both measured within subject, will need you to specify 3x3 cells in the input box below. Does this clarify the analysis?

Kind regards Johnny

keslin527 commented 9 months ago

RM_ANOVA_image

Thanks for replying so quickly. Can you all see the image now? Johnny-your graphic appears to be two within IV's (independent variables; speed & alcohol consumption) and 1 between IV (ambient light, as expressed by "daytime") on some outcome (what is it, by the way?). So the Repeated Measures Factors box is great for adding the two within IV's.

What I am asking is, what if there is another DV (outcome/dependent variable)? There would be 9 more columns in the dataset for each of the levels for speed and alcohol. It appears that adding these extra 9 rows into the Repeated Measures Factors box would just be read as 9 more contexts of the variable, rather than as a separate variable. Is there a way in this module to distinguish between two separate outcomes?

tomtomme commented 9 months ago

@keslin527 Yes, johnny named the 2 RM facors like IVs. Both RM Factors are splittet in 3 time steps. And both could measure different DVs at those times - the animation just does not show the names of the DV(s). So Johnny also could just have named those two factors like the DVs that are measured at the 3 time steps. So this makes no difference. If you put >1 RM factors in, you can also accomodate >1 DV at the same time.

You could for example measure the two DVs systolic and diastolic from your picture. And you could measure it for Johnys speed and alcohol consumption IVs.

Assuming that the data is in wide format you would then need the following columns to accomadate 4 Factors with the 2 IVs and the 2 DVs:

RM Factor 1

speed_systolic at time 1 speed_systolic at time 2 speed_systolic at time 3

RM Factor 2

speed_diastolic at time 1 speed_diastolic at time 2 speed_diastolic at time 3

RM Factor 3

alc_systolic at time 1 alc_systolic at time 2 alc_systolic at time 3

RM Factor 4

alc_diastolic at time 1 alc_diastolic at time 2 alc_diastolic at time 3

So this results in 4 RM Factors. Put all 4 in the RM ANOVA module and you essentially get a RM MANOVA over 2 within IVs and DVs. Right?

But maybe you are thinking in a "long format" data structure instead of "wide format" one. Then the data layout would look different then described above. At the moment however JASPs RM ANOVA module needs the data to come in wide format.

P.S. I hope my above solution is possible with current implementation of jasp. I could not find however a dataset in jasps data library which does this.

keslin527 commented 9 months ago

I apologize for not explaining myself more clearly. Thank you for your patience!

The way Johnny has his setup appears to be a 2-way (3x3) ANOVA. Obviously, I don’t have his dataset, so I’ll try to explain in more detail using mine. For my research question, I’d like to look at the effect of a pre/post intervention of systolic and diastolic blood pressure. First I’ll demonstrate your proposed analysis with the output on SPSS and JASP. I’ll then demonstrate the JASP analysis I am asking for, on SPSS. In SPSS, the way you propose looks like this with one factor (time) and one outcome (blood pressure).

image

As demonstrated below, the data are in wide format with pre- and post-systolic pressures pasted in the “Within-Subjects Variables" box below. This is analogous to the "Repeated Measures Cells" in JASP. We can ask for post hoc tests, effect sizes, etc. which I will not post on here.

image

After we do this, we get the output below with pre-and post-blood pressures as each of 4 contexts of a pressure variable. However, what if I wanted to look at other variables like cholesterol, this would not work because the software is looking at each variable as a context rather than as a separate variable! Additionally, SPSS will offer a Multivariate Tests table which appears to be a sort of profile analysis. Does JASP currently offer this feature for the Repeated Measures ANOVA suite?

image

SPSS then offers Univariate tests under Tests of within-Subjects Effects/Contrasts as shown below.

image

image

These are the same outputs offered by JASP as shown in the pictures below. Again, this is not appropriate because we have different variables being compared as if they are just 4 different contexts of 1 variable, when we are trying to look at it as 2 contexts (pre + post) of 2 separate variables (systolic BP and diastolic BP). What if we also had cholesterol, blood sugar etc.? These are all different variables that should not be compared together in a post hoc test. Therefore, this manner of analysis is improper.

JASP setup:

image

JASP Output:

image

The feature I am requesting is below, as demonstrated on SPSS. Again, I’m looking for the multivariate result for two levels on time for two separate outcomes (systolic and diastolic). Potentially I may want to add k more outcomes to the analysis (cholesterol, blood sugar, etc). In SPSS these would be added to the Measure Name box, and each variable is labeled separately as shown below. The Measure Name box is where separate dependent variables should be added in order to let the software know we’re analyzing these as separate outcomes. Is this already possible in JASP? If not, would it be possible to add it?

image

Once we do this, then we can add the two levels for systolic pressure, and diastolic pressure as shown below. Again, note these data are in wide format.

image

image

The outputs are shown below. It lists the two DV’s and offers descriptives for each. It also offers a multivariate test output with a p-value and partial eta squared for the multivariate output. Mauchly’s test of sphericity then evaluates the two (or more) outcome variables separately. You will note SPSS will repeat the multivariate output several times.

image

Following the multivariate output, it offers univariate output separate for each outcome as shown below.

image

Finally, post hoc tests for these univariate ANOVA’s are offered as shown in the Pairwise Comparisons box below. The post hoc tests evaluates each variable separately. This is more preferable than the first example where variables measured differently are all in the same post hoc test.

image

What I am asking is can you implement the following features into JASP?

  1. Adding a Measure Name (i.e., dependent variable) box where the user can differentiate between the k different dependent variables. Adding them to the Repeated Measures Factors box in JASP is not appropriate, because the software just sees it as a RM ANOVA with different contexts instead of different outcomes.
  2. A multivariate test for the Repeated Measures MANOVA?

OR

If it already does this, would you be able to explain how?

Thank you once again for your time and patience.

tomtomme commented 9 months ago

  1. I tried to explain this above. Jhonny too, I think. From my perspective JASPs Repeated Measures Factors box is appropriate. Why do you think that JASP treats different Factors as only one DV? But maybe you have not recognized this? grafik This is where you would initialize and name the second DV. Also the post hoc tests then show separate tables for those DVs (and tables for interactions) Here is an example file with such an analysis from jasps data library: AlcoholAttitudes.jasp.zip Remove the .zip before opening.

  2. This one is a bit more tricky in JASP. You would need to convert your data from wide to long. We are working on GUI elements for the data edit mode, to make this easy. But I can give no timeline for when this will be done. So at the moment you would need to convert from wide to long e.g. in excel. After that you could use the MANOVA module to get Pillays Trace etc. The problem with this solution: I do not know if MANOVA and RM MANOVA rely on the exact same assumptions. Would be cool if you test this with your data and give me feedback, if this gives you the same results as in SPSS.

JohnnyDoorn commented 9 months ago

To add to @tomtomme , we do not provide these multivariate tests (MANOVA is not very high on our priority list unfortunately), but I think the most important tests for these types of research scenario's are captured by RM ANOVA. Here is a video to create the structure that (I think) you are looking for:

https://github.com/jasp-stats/jasp-issues/assets/15704203/833447f8-a150-4dd5-856f-d6ac31927dda

keslin527 commented 9 months ago

I haven't had a chance to use the alcohol dataset as yet, but I can say off the bat that MANOVA and RM MANOVA do not share the same assumptions. Asking for Pillai's Trace through the MANOVA route would be incorrect, as using regular MANOVA with repeated measures data violates the assumption of the independence of observations (which is why RM MANOVA is necessary). This inflates Type I error.

I did run the blood pressure dataset the way Johnny suggested, and the outputs still differ from that of SPSS; I am not sure that adding the outcomes as a factor is the correct way to do it.

That said, I very much appreciate the honest response that MANOVA is not a priority for the team at this time; JASP is free and I suppose that beggars cannot be choosers. That said, if it ever does become a priority once again, it would be very helpful for intervention studies in health sciences studies as RM MANOVA is a powerful tool in these areas. I've also taken the liberty to attach a paper for that purpose; I hope that you do not mind.

Thank you so much again for your help and patience. Have a great week!

MANOVA Method for Analyzing Repeated Measures Designs_An Extensive Primer.pdf

tomtomme commented 9 months ago

A paper is always helpful. Also all the details you added to the conversation. So thank you and lets hope that someone comes along to implement this!

keslin527 commented 9 months ago

No problem! Here's another link in case this helps anyone reading this thread who may be able to help:

https://statistics.laerd.com/spss-tutorials/one-way-repeated-measures-manova-using-spss-statistics.php

tomtomme commented 2 months ago

Summing up everything about RM MANOVA so far:

Rationale:

Examples:

Thus we need a new RM MANOVA module or more features within the RM ANOVA module like:

Material:

keslin527 commented 2 months ago

I appreciate the update; I was unaware (though not surprised) that others requested the feature. The feature's use in the field of agriculture is also duly noted.

Thank you.