[Feature Request]: Crossed contrasts in multi-factor multi-level ANOVA

[bfmalle]

Description

Multi-factorial multi-level ANOVAs are difficult to interpret unless one defines meaningful contrasts for each main effect AND crosses these contrasts for the interaction term. Currently, JASP's interaction contrasts treat the cells of the interaction as if they were one big factor. Instead, we should be able to cross the defined contrasts and be able to interpret the "sub-interactions"

Purpose

Meaningful contrasts to interpret complex designs

Use-case

Any multi-factorial design where at least one factor has more than one df

Is your feature request related to a problem?

The problem is that we cannot currently break down interactions (e.g., in a 3 x 3 design) into meaningful patterns. This isn't a statistical problem but a practical, interpretational problem.

Is your feature request related to a JASP module?

ANOVA

Describe the solution you would like

We currently are able to define contrasts for each factor in a design (e.g., 3 x 2), but these contrasts are only useful for testing the main effects of these factors. If we want to "apply" the contrasts to the (say, 3 x 2) interaction term, JASP doesn't cross the defined contrasts (A1 x B, A2 x B) but asks us to define a new contrasts that then treats all 2 x 3 cells as if the were one big 1 x 6 factor.

Describe alternatives that you have considered

Allow user do define a contrast for each factor in the model and then automatically cross these contrasts for the interaction terms. For example, if A has 3 levels and B has 2 levels, then we might define two Helmert contrasts for A (A1v23, A2v3) and B is of course just a difference contrast B1v2). The interaction would then consist of two crossed terms: A1v23 x B12 and A2v3 x B12. These are meaningful interaction contrasts that can be easily tested, graphed, and interpreted.

Additional context

I am attaching a zipped JASP file for a 3 x 3 design, which produces the strange "interaction contrasts." I also include a printout of an SPSS output in which I defined Helmert contrasts for one (3-level) factor and Polynomial contrasts for the other (3-level factor), and the program automatically crosses the two contrasts so that I have 4 "subcontrasts." 3 x 3 design example.jasp.zip 3x3ANOVA with crossed contrasts SPSS.pdf

[tomtomme]

@bfmalle I am unsure, if I understand the problem. Your file has a 3x3 design that enable 9 pairwise comparisons - this is what JASPs produces. But since you produced Helmert contrasts for the two main effects, this effectively reduces the available comparisons to a 2x2 matrix. This is what SPSS puts out in the last table seen in the PDF. Is this correct so far? So to implement this, our interaction contrast table would need to be conditional on the tables produced for the main effect contrasts.

@JohnnyDoorn Any idea if this is feasible? Also: Could @bfmalle use custom contrasts as a workaround?

[bfmalle]

Yes, this is correct. There are only (3-1) x (3-1) = 4 dfs for the interaction contrasts, represented by your "2 x 2" matrix. Pairwise comparisons don't test the interactions. Interactions are logically tied to the main effects, so ideally they would "continue" the contrasts defined for the main effects.

I have been using custom contrasts, but honestly, it's a pain, can lead to inaccuracies, and gets worse with a 3-factor design...

[JohnnyDoorn]

Hi @bfmalle,

Thanks for the detailed request - this would be great to have. If I understand correctly, it's a sort of "simple contrasts" - like "simple main effects" is an anova conditional on each level of a factor, this would be contrast analysis for each (contrast-)level of a factor? I will look into this because I think it's one of the last "bigger" features from SPSS that is not in JASP's ANOVA yet.

I'm having some trouble wrapping my head around the SPSS output though, especially the last table - could you help me interpret some of its metrics? For instance the first row: parameter = 2 coefficient = 7.9667364014 SE = 1.63217 t =- 4.88106 p = .00001 What does this particular t/p-value tell me? Or am i I even looking at the wrong table here?

[bfmalle]

No, it's the regular set of crossed interaction contrast, as shown in the blue and green columns below. And the parameter table shows the first two contrasts (main effects for Dose), tested for significance. These contrasts are also shown in the ANOVA table, as the individual single-df tests for Dose. These are the same as the ones Jasp shows in the Contrast table for Dose. What's missing in Jasp are the tests for the four interaction contrasts, shown in the second image below (with green corresponding to the four green contrast weight schemes in the first image). Confusingly, SPSS numbers the "parameters" differently in the ANOVA table and in the individual contrast (t) tests.

[bfmalle]

Hmm, the second image wasn't displayed in the previous email. Here it is:

[bfmalle]

And, as a reminder, the ANOVA table is here. You see how the four tests correspond precisely to each other (where F = t^2)

[tomtomme]

@bfmalle Thanks for the further details.