search

pitakakariki / simr

Power Analysis of Generalised Linear Mixed Models by Simulation
70 stars 20 forks source link

Using SIMR for a categorical variable with 7 levels #163

Open eahq opened 5 years ago

eahq commented 5 years ago

Hello, I have a multinational dataset and am trying to conduct a post-hoc power analysis for a model with 1 categorical variable with 7 levels, and three covariates (with a random effect for country). I have successfully estimated power for a simpler model without the categorical predictor (94%), but I am struggling to figure out how to calculate power for the categorical predictor. I can't ascribe a fixed effect to the categorical predictor, I assume because of the multiple levels. I have tried: powerSim(model, fixed("categorical variable", "lr"), nsim = 50), and I get 100% power. Any help would be hugely appreciated.

pitakakariki commented 5 years ago

Are you getting any errors? The number of levels shouldn't be a problem.

Try making the fixed effects smaller with fixef<- and see if that reduces the estimated power.

Also: be careful doing post-hoc power analysis. It's often a bad idea and usually confidence intervals will be easier to interpret.

eahq commented 5 years ago

I think the major issue I'm having is assigning the categorical variable an effect size. My categorical variable is "Culture". I can run powerSim(model, fixed("Culture", "lr"), nsim = 50), but I can't run: fixef(model)["Culture"] <- 0.05. I get an error: "Culture" is not the name of a fixed effect.

I believe this issue is coming up because while I specify "Culture" as a fixed effect in my model, the results only include fixed effects for the dummy codes, like "Culture_Asia".

I have tried assigning a fixed effect to one of the dummy coded comparisons, e.g, fixef(model)["Culture_Asia"] <- 0.05, and running powerSim(model, fixed("Culture_Asia", "z"), nsim = 50). I get a predicted power of 12%.

Is this correct?

I'm wondering if I would be better to calculate power for detecting a difference between a simpler model (only demographics) to a model that includes culture?

pitakakariki commented 5 years ago

Use fixef(model) to see the names of the fixed effects you can assign. You should probably specify appropriate effects for all levels.

eahq commented 5 years ago

Ah ok, thanks. So is it possible to assign an effect size to a categorical variable with multiple levels simultaneously? In the Test examples page, there is:

doTest(gm1, fixed("period", "lr"))

And I would be interested in assigning a fixed effect to something like "period" to run a power analysis for all levels of the categorical predictor.

From: Peter Green notifications@github.com Sent: Sunday, September 8, 2019 8:22 PM To: pitakakariki/simr simr@noreply.github.com Cc: Emily Harris emily.harris@queensu.ca; Author author@noreply.github.com Subject: Re: [pitakakariki/simr] Using SIMR for a categorical variable with 7 levels (#163)

Use fixef(model) to see the names of the fixed effects you can assign. You should probably specify appropriate effects for all levels.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpitakakariki%2Fsimr%2Fissues%2F163%3Femail_source%3Dnotifications%26email_token%3DANDKJ2FYHIA6EGFV3RDICJLQIWJNPA5CNFSM4IUCNCP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6F6DDY%23issuecomment-529260943&data=02%7C01%7Cemily.harris%40queensu.ca%7C56a89b74d09c4808b12408d734bbd7a8%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C637035853708745307&sdata=LLEZF20tctpcJZwmi0QaxdQ6xIZeAe3mJG1k2UeIYt4%3D&reserved=0, or mute the threadhttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FANDKJ2C33EJBZELNNZ7C4YLQIWJNPANCNFSM4IUCNCPQ&data=02%7C01%7Cemily.harris%40queensu.ca%7C56a89b74d09c4808b12408d734bbd7a8%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C637035853708755302&sdata=6ESYfTsOPjSS8DhflJNmb292YcqoitebDq0u1rUMw1w%3D&reserved=0.