search

easystats / performance

:muscle: Models' quality and performance metrics (R2, ICC, LOO, AIC, BF, ...)
https://easystats.github.io/performance/
GNU General Public License v3.0
965 stars 87 forks source link

r2_nakagawa and glmmTMB with beta_family #710

Closed POHedwall closed 2 weeks ago

POHedwall commented 2 months ago

Hi everyone, Last year I was working on some hurdle glmmTMB models with beta_family and produced pseudo-r2s from these models with r2_nakagawa. After a couple of months´ break, including a shift of computer (now Windows10) and installing latest versions of R and packages, I now get considerably different results from r2_nakagawa including a warning:

"r2_nakagawa(GLMM2.6)

R2 for Mixed Models

Conditional R2: 1.086 Marginal R2: 0.943 Warning message: Model's distribution-specific variance is negative. Results are not reliable."

Last time I used the code the corresponding conditional and marginal R2s were 0.765 and 0.66 which seems more reasonable. No warning that time either.

Unfortunately I dont remember which version of R, glmmTMB and Performance I used previously but I wonder if any updates could have caused this difference, or if there are other known reasons? Attach code and data, please let me know if anything else is needed.

Grateful for any help on this matter! Per-Ola Hedwall

mydata.csv code_example.txt

strengejacke commented 1 month ago

Yes, there have been changes both in glmmTMB in how the variance is calculated, and some revisions in insight::get_variance() (which is responsible for the R2 calculation for mixed models). This is probably not easy to resolve, but we're aware of these issues (see https://github.com/easystats/insight/issues/664 or https://github.com/easystats/insight/issues/764, to mention two examples).

strengejacke commented 1 month ago

See also https://github.com/easystats/insight/issues/681 and https://github.com/easystats/insight/issues/679

strengejacke commented 2 weeks ago

It should be fixed now. The current result is:

# R2 for Mixed Models

  Conditional R2: 1.000
     Marginal R2: 0.868

which is in line with MuMIn::r.squaredGLMM(). Package updates are not yet available, but will be soon.

POHedwall commented 2 weeks ago

Great, thanks a lot!