easystats / performance

:muscle: Models' quality and performance metrics (R2, ICC, LOO, AIC, BF, ...)
https://easystats.github.io/performance/
GNU General Public License v3.0
965 stars 87 forks source link

check_model failing on logistic regression #701

Closed bbolker closed 3 months ago

bbolker commented 3 months ago

It's possible that this will get sorted out as part of the process of checking/validating #698, but I thought I would report it in case it gets missed.

## remotes::install_github("easystats/performance")
## remotes::install_github("easystats/performance", ref = remotes::github_pull(698))
mm <- transform(mtcars, bigcyl = as.numeric(cyl > 2))
m <- glm(bigcyl ~ mpg, data = mm, family = binomial)
performance::check_model(m)

This triggers an error from ggplot2,

dim(data) must return an of length 2.

At this point data is:

 class(data)
[1] "performance_simres"     "see_performance_simres" "DHARMa"                
Browse[2]> where
where 1: .prevalidate_data_frame_like_object(data)
where 2: validate_as_data_frame(model)
where 3: withCallingHandlers(expr, condition = function(cnd) {
    {
        .__handler_frame__. <- TRUE
        .__setup_frame__. <- frame
        if (inherits(cnd, "message")) {
            except <- c("warning", "error")
        }
        else if (inherits(cnd, "warning")) {
            except <- "error"
        }
        else {
            except <- ""
        }
    }
    while (!is_null(cnd)) {
        if (inherits(cnd, "error")) {
            out <- handlers[[1L]](cnd)
            if (!inherits(out, "rlang_zap")) 
                throw(out)
        }
        inherit <- .subset2(.subset2(cnd, "rlang"), "inherit")
        if (is_false(inherit)) {
            return()
        }
        cnd <- .subset2(cnd, "parent")
    }
})
where 4: doTryCatch(return(expr), name, parentenv, handler)
where 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
where 6: tryCatchList(expr, classes, parentenv, handlers)
where 7: tryCatch(withCallingHandlers(expr, condition = function(cnd) {
    {
        .__handler_frame__. <- TRUE
        .__setup_frame__. <- frame
        if (inherits(cnd, "message")) {
            except <- c("warning", "error")
        }
        else if (inherits(cnd, "warning")) {
            except <- "error"
        }
        else {
            except <- ""
        }
    }
    while (!is_null(cnd)) {
        if (inherits(cnd, "error")) {
            out <- handlers[[1L]](cnd)
            if (!inherits(out, "rlang_zap")) 
                throw(out)
        }
        inherit <- .subset2(.subset2(cnd, "rlang"), "inherit")
        if (is_false(inherit)) {
            return()
        }
        cnd <- .subset2(cnd, "parent")
    }
}), stackOverflowError = handlers[[1L]])
where 8: try_fetch(validate_as_data_frame(model), error = function(cnd) cli::cli_abort(glue(msg0), 
    parent = cnd))
where 9: fortify.default(data, ...)
where 10: fortify(data, ...)
where 11: ggplot.default(x, ggplot2::aes(x = .data$x, y = .data$y))
where 12: ggplot2::ggplot(x, ggplot2::aes(x = .data$x, y = .data$y))
where 13: .plot_diag_qq(x$QQ, size_point, size_line, alpha_level = alpha_level, 
    detrend = detrend, theme_style = style, colors = colors, 
    dot_alpha_level = dot_alpha_level, show_dots = TRUE, model_info = model_info, 
    model_class = model_class)
where 14: plot.see_check_model(x, style = style, colors = colors, type = type, 
    n_colums = n_columns, ...)
where 15: NextMethod()
where 16: plot.check_model(x, style = style, colors = colors, type = type, 
    n_colums = n_columns, ...)
where 17: graphics::plot(x, style = style, colors = colors, type = type, 
    n_colums = n_columns, ...)
where 18: withCallingHandlers(expr, message = function(c) if (inherits(c, 
    classes)) tryInvokeRestart("muffleMessage"))
where 19: suppressMessages(graphics::plot(x, style = style, colors = colors, 
    type = type, n_colums = n_columns, ...))
where 20: withCallingHandlers(expr, warning = function(w) if (inherits(w, 
    classes)) tryInvokeRestart("muffleWarning"))
where 21: suppressWarnings(suppressMessages(graphics::plot(x, style = style, 
    colors = colors, type = type, n_colums = n_columns, ...)))
where 22: print.see_check_model(x)
where 23: NextMethod()
where 24: print.check_model(x)
where 25: (function (x, ...) 
UseMethod("print"))(x)
where 26: check_model.default(m)
where 27: performance::check_model(m)
strengejacke commented 3 months ago

Can you run easystats::install_latest(force = TRUE) and then retry? For me, everything is ok.

mm <- transform(mtcars, bigcyl = as.numeric(cyl > 2))
m <- glm(bigcyl ~ mpg, data = mm, family = binomial)
performance::check_model(m)

Session ino ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.3.3 (2024-02-29 ucrt) #> os Windows 11 x64 (build 22631) #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate German_Germany.utf8 #> ctype German_Germany.utf8 #> tz Europe/Berlin #> date 2024-03-25 #> pandoc 3.1.1 @ C:/Users/DL/AppData/Local/Pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> bayestestR 0.13.2 2024-03-12 [1] https://easystats.r-universe.dev (R 4.3.3) #> benchmarkme 1.0.8 2022-06-12 [1] CRAN (R 4.3.0) #> benchmarkmeData 1.0.4 2020-04-23 [1] CRAN (R 4.3.0) #> bitops 1.0-7 2021-04-24 [1] CRAN (R 4.3.0) #> boot 1.3-30 2024-02-26 [1] CRAN (R 4.3.3) #> caTools 1.18.2 2021-03-28 [1] CRAN (R 4.3.0) #> cli 3.6.2 2023-12-11 [1] CRAN (R 4.3.2) #> codetools 0.2-19 2023-02-01 [2] CRAN (R 4.3.3) #> colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.3.0) #> datawizard 0.9.1.8 2024-03-23 [1] https://easystats.r-universe.dev (R 4.3.3) #> DEoptimR 1.1-3 2023-10-07 [1] CRAN (R 4.3.1) #> DHARMa 0.4.6 2022-09-08 [1] CRAN (R 4.3.0) #> digest 0.6.35 2024-03-11 [1] CRAN (R 4.3.3) #> doParallel 1.0.17 2022-02-07 [1] CRAN (R 4.3.0) #> dplyr 1.1.4 2023-11-17 [1] CRAN (R 4.3.2) #> evaluate 0.23 2023-11-01 [1] CRAN (R 4.3.2) #> fansi 1.0.6 2023-12-08 [1] CRAN (R 4.3.2) #> farver 2.1.1 2022-07-06 [1] CRAN (R 4.3.0) #> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0) #> foreach 1.5.2 2022-02-02 [1] CRAN (R 4.3.0) #> fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.1) #> generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.0) #> ggplot2 3.5.0 2024-02-23 [1] CRAN (R 4.3.3) #> ggrepel 0.9.5 2024-01-10 [1] CRAN (R 4.3.2) #> glue 1.7.0 2024-01-09 [1] CRAN (R 4.3.2) #> gtable 0.3.4 2023-08-21 [1] CRAN (R 4.3.1) #> htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.3.2) #> httr 1.4.7 2023-08-15 [1] CRAN (R 4.3.1) #> insight 0.19.10 2024-03-22 [1] https://easystats.r-universe.dev (R 4.3.3) #> iterators 1.0.14 2022-02-05 [1] CRAN (R 4.3.0) #> knitr 1.45 2023-10-30 [1] CRAN (R 4.3.1) #> labeling 0.4.3 2023-08-29 [1] CRAN (R 4.3.1) #> lattice 0.22-5 2023-10-24 [1] CRAN (R 4.3.3) #> lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.3.2) #> lme4 1.1-35.1 2023-11-05 [1] CRAN (R 4.3.2) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0) #> MASS 7.3-60.0.1 2024-01-13 [1] CRAN (R 4.3.2) #> Matrix 1.6-5 2024-01-11 [1] CRAN (R 4.3.2) #> memuse 4.2-3 2023-01-24 [1] CRAN (R 4.3.0) #> mgcv 1.9-1 2023-12-21 [1] CRAN (R 4.3.2) #> minqa 1.2.6 2023-09-11 [1] CRAN (R 4.3.1) #> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.3.0) #> nlme 3.1-164 2023-11-27 [1] CRAN (R 4.3.2) #> nloptr 2.0.3 2022-05-26 [1] CRAN (R 4.3.0) #> opdisDownsampling 0.8.3 2023-12-13 [1] CRAN (R 4.3.2) #> patchwork 1.2.0 2024-01-08 [1] CRAN (R 4.3.2) #> performance 0.11.0 2024-03-24 [1] https://easystats.r-universe.dev (R 4.3.3) #> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.0) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0) #> pracma 2.4.4 2023-11-10 [1] CRAN (R 4.3.2) #> purrr 1.0.2 2023-08-10 [1] CRAN (R 4.3.1) #> qqconf 1.3.2 2023-04-14 [1] CRAN (R 4.3.0) #> qqplotr 0.0.6 2023-01-25 [1] CRAN (R 4.3.0) #> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.0) #> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.0) #> R.oo 1.26.0 2024-01-24 [1] CRAN (R 4.3.2) #> R.utils 2.12.3 2023-11-18 [1] CRAN (R 4.3.2) #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0) #> Rcpp 1.0.12 2024-01-09 [1] CRAN (R 4.3.2) #> reprex 2.1.0 2024-01-11 [1] CRAN (R 4.3.2) #> rlang 1.1.3 2024-01-10 [1] CRAN (R 4.3.2) #> rmarkdown 2.26 2024-03-05 [1] CRAN (R 4.3.3) #> robustbase 0.99-2 2024-01-27 [1] CRAN (R 4.3.2) #> scales 1.3.0 2023-11-28 [1] CRAN (R 4.3.2) #> see 0.8.3.1 2024-03-24 [1] https://easystats.r-universe.dev (R 4.3.3) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0) #> styler 1.10.2 2023-08-29 [1] CRAN (R 4.3.1) #> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.0) #> tidyselect 1.2.1 2024-03-11 [1] CRAN (R 4.3.3) #> twosamples 2.0.1 2023-06-23 [1] CRAN (R 4.3.1) #> utf8 1.2.4 2023-10-22 [1] CRAN (R 4.3.1) #> vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.3.2) #> withr 3.0.0 2024-01-16 [1] CRAN (R 4.3.2) #> xfun 0.42 2024-02-08 [1] CRAN (R 4.3.2) #> yaml 2.3.8 2023-12-11 [1] CRAN (R 4.3.2) #> #> [1] C:/Users/DL/AppData/Local/R/win-library/4.3 #> [2] C:/Program Files/R/R-4.3.3/library #> #> ────────────────────────────────────────────────────────────────────────────── ```

Created on 2024-03-25 with reprex v2.1.0

bbolker commented 3 months ago

Must have been a transient version-mismatch thing. Thanks for checking.