pitakakariki / simr

Power Analysis of Generalised Linear Mixed Models by Simulation
68 stars 19 forks source link

Power is always 0% #257

Open lovelyjerry666 opened 1 year ago

lovelyjerry666 commented 1 year ago

When I was doing Power analysis, the result of Power was always 0%. I don't know where the problem is? Here is my code and results My model: fit4=lmer (data=Q6, rt~sf * sr+(1 | sub)+(1+sr | item))

results:

powerSim(fit4,test = fixed("sf"),nsim = 10)

Power for predictor 'sf', (95% confidence interval):=================================================================| 0.00% ( 0.00, 30.85)

Test: Likelihood ratio

Based on 10 simulations, (10 warnings, 10 errors) alpha = 0.05, nrow = 3191

Time elapsed: 0 h 0 m 7 s

nb: result might be an observed power calculation Warning message: In observedPowerWarning(sim) : This appears to be an "observed power" calculation

powerSim(fit4,test = fixed("sf"),nsim = 500)

Simulating: |=====================================================================Simulating: |=====================================================================Power for predictor 'sf', (95% confidence interval):=================================================================| 0.00% ( 0.00, 0.74)

Test: Likelihood ratio

Based on 500 simulations, (500 warnings, 500 errors) alpha = 0.05, nrow = 3191

Time elapsed: 0 h 5 m 15 s

nb: result might be an observed power calculation Warning message: In observedPowerWarning(sim) : This appears to be an "observed power" calculation

I have reviewed the error report:

lastResult()$errors stage index message 1 Testing 1 Test did not return a p-value 2 Testing 2 Test did not return a p-value 3 Testing 3 Test did not return a p-value 4 Testing 4 Test did not return a p-value 5 Testing 5 Test did not return a p-value 6 Testing 6 Test did not return a p-value 7 Testing 7 Test did not return a p-value 8 Testing 8 Test did not return a p-value 9 Testing 9 Test did not return a p-value 10 Testing 10 Test did not return a p-value

I changed the test to Z test, but the result is still 0%, but the error was reported and changed to "Testing 1 Subscript out of bounds“.

powerSim(fit4,test = fixed("sf","z"),nsim = 10)

Power for predictor 'sf', (95% confidence interval):=================================================================| 0.00% (0.00, 30.85)

Test: z-test

Based on 10 simulations, (0 warnings, 10 errors) alpha = 0.05, nrow = 3191

Time elapsed: 0 h 0 m 5 s

nb: result might be an observed power calculation Warning message: In observedPowerWarning(sim) : This appears to be an "observed power" calculation

lastResult()$errors

stage index message 1 Testing 1 Subscript out of bounds 2 Testing 2 Subscript out of bounds 3 Testing 3 Subscript out of bounds 4 Testing 4 Subscript out of bounds 5 Testing 5 Subscript out of bounds 6 Testing 6 Subscript out of bounds 7 Testing 7 The subscript is out of bounds 8 Testing 8 Subscript out of bounds 9 Testing 9 Subscript out of bounds 10 Testing 10 The subscript is out of bounds

pitakakariki commented 1 year ago

Bit of a puzzle. Could you please try print(fit4) and doTest(fit4, fixed("sf")) to see if they provide any clues?

lovelyjerry666 commented 1 year ago

Bit of a puzzle. Could you please try print(fit4) and doTest(fit4, fixed("sf")) to see if they provide any clues?

I executed this command and the result is as follows:

print(fit4) Linear mixed model fit by REML ['lmerModLmerTest'] Formula: rt ~ sf * sr + (1 | sub) + (1 + sr | item) Data: Q6 REML criterion at convergence: 41257.32 Random effects: Groups Name Std.Dev. Corr item (Intercept) 55.50
sr1 19.24 -0.14 sub (Intercept) 101.72
Residual 149.24
Number of obs: 3191, groups: item, 54; sub, 34 Fixed Effects: (Intercept) sf1 sr1 sf1:sr1
764.9745 -16.0918 -18.0205 -0.3608

My two variables 'sf' and 'sr' are both factor variables, and I found that I mistakenly wrote sf1 as sf. After making the changes, it can run normally

powerSim(fit4,test = fixed("sf1:sr1","z"),nsim = 50)

Power for predictor 'sf1:sr1', (95% confidence interval):============================================================| 6.00% ( 1.25, 16.55)

Test: z-test Effect size for sf1:sr1 is -0.36

Based on 50 simulations, (1 warning, 0 errors) alpha = 0.05, nrow = 3191

Time elapsed: 0 h 0 m 12 s

nb: result might be an observed power calculation

Because there is not much knowledge in this field, I do not know whether this is the reason