search

pitakakariki / simr

Power Analysis of Generalised Linear Mixed Models by Simulation
68 stars 19 forks source link

Power analysis for main effect with a significant interaction #198

Open avdvloet opened 3 years ago

avdvloet commented 3 years ago

Hi,

I want to perform a power analysis for a mixed linear model (lmer, lme4 package) of the following type:

continuous variable ~ discrete variable a * discrete variable b + (1| discrete variable c: discrete variable b)

If I run the appropriate anova on it, I get significance for the interaction term. Yet, I am interested in the power for a significant effect size for one of the levels of discrete variable a. In other words I would like to run the power analysis for a main effect, while my model contains a significant interaction.

First of all, is this possible? If yes, I've read some things about it in the appendix S1 of your paper, using e.g. doTest(fm1, fcompare(~ recipe + temp)) in one of the examples there. What is the link with running a PA on a main effect? How should I use this function for my case and how should interpret this in using it for my power analysis?

Thanks in advance!

pitakakariki commented 3 years ago

It sounds like you want this paragraph from the Test examples vignette:

If you were (for some reason) especially interested in the significance for the dummy variable period2 you could use a z-test. This test uses the value Pr(>|z|) reported in the summary above.

doTest(gm1, fixed("period2", "z"))

avdvloet commented 3 years ago

Sorry, but I still don't know what is meant by that.

It says on that page that you shouldn't test main effects when they are in a significant interaction term, which in the case for my model. But then what is the solution, can I still do it in some way? Or is the doTest function just used to get the effect size you then want to use for your power analysis?

pitakakariki commented 3 years ago

simr shouldn't have any problem testing the main effect despite the significant interaction. That advice is more of a generally best-practice thing but there are exceptions so if that test makes sense for your model it should still work.