easystats / performance

:muscle: Models' quality and performance metrics (R2, ICC, LOO, AIC, BF, ...)
https://easystats.github.io/performance/
GNU General Public License v3.0
1.03k stars 94 forks source link

`check_zeroinflation()` not working with glmmTMB? #744

Closed sj-perry closed 4 months ago

sj-perry commented 4 months ago

Hello,

In creating some teaching materials, I think I've found a potential bug in the check_zeroinflation() function. At the very least, the # of predicted zeros does not align with the visualization from check_predictions() run on the same model.

Here is a reproducible example:

set.seed(1)

# Creating 'dummy' variable
x <- rep(c(0, 1), each = 50)

# Generate counts
y <- rpois(100, 0.25 + 2*x)

df <- data.frame(x, y)

# Fitting intercept-only model
m1 <- glm(y ~ 1,
          data = df,
          family = poisson())

check_predictions(m1)

plot1

# output of check_zeroinflation aligns with check_predictions
check_zeroinflation(m1)
# Check for zero-inflation

   Observed zeros: 43
  Predicted zeros: 30
            Ratio: 0.70

Model is underfitting zeros (probable zero-inflation).

So this all seems to be working as intended. However, I then fit a model with zeroinflation using glmmTMB, and the plot looks good.

# Fitting zero-inflated model with glmmTMB
m2 <- glmmTMB(y ~ 1,
              data = df,
              family = poisson(),
              ziformula = ~ 1)

# This seems to be working?
check_predictions(m2)

plot2

However, the check_zeroinflation() part does not change compared to a poission model. So I guess the zero-inflated component is taken into account in the predictions but not when checking for zeroinflation specifically?

check_zeroinflation(m2)
# Check for zero-inflation

   Observed zeros: 43
  Predicted zeros: 30
            Ratio: 0.70

Model is underfitting zeros (probable zero-inflation).
strengejacke commented 4 months ago

Are you probably using old package versions? I get:

set.seed(1)

# Creating 'dummy' variable
x <- rep(c(0, 1), each = 50)

# Generate counts
y <- rpois(100, 0.25 + 2*x)

df <- data.frame(x, y)
m2 <- glmmTMB::glmmTMB(y ~ 1,
  data = df,
  family = poisson(),
  ziformula = ~1
)
#> Warning in checkDepPackageVersion(dep_pkg = "TMB"): Package version inconsistency detected.
#> glmmTMB was built with TMB version 1.9.12
#> Current TMB version is 1.9.14
#> Please re-install glmmTMB from source or restore original 'TMB' package (see '?reinstalling' for more information)

performance::check_zeroinflation(m2)
#> # Check for zero-inflation
#> 
#>    Observed zeros: 43
#>   Predicted zeros: 43
#>             Ratio: 0.99
#> Model seems ok, ratio of observed and predicted zeros is within the
#>   tolerance range (p > .999).

Created on 2024-07-08 with reprex v2.1.1

Try to update all packages, and also easystats::install_latest() (assuming you have the easystats package installed).

roaldarbol commented 4 months ago

Just tested this too, and works fine on my end as well.