easystats / performance

:muscle: Models' quality and performance metrics (R2, ICC, LOO, AIC, BF, ...)
https://easystats.github.io/performance/
GNU General Public License v3.0
965 stars 87 forks source link

CRAN Release 0.11.0 #700

Closed strengejacke closed 3 months ago

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 96.15385% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 60.26%. Comparing base (01eff88) to head (cc1342d).

Files Patch % Lines
R/check_model_diagnostics.R 0.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #700 +/- ## ========================================== + Coverage 59.88% 60.26% +0.37% ========================================== Files 86 86 Lines 6252 6274 +22 ========================================== + Hits 3744 3781 +37 + Misses 2508 2493 -15 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

strengejacke commented 3 months ago

@easystats/maintainers I get strange errors when I check on win-builder for win-devel. This also occurs during CRAN submission, so it's not just a win-builder issue. And, this does not happen for linux-devel.

When I disable parallel testing, I get this error:

Running the tests in 'tests/testthat.R' failed.
Complete output:
  > library(testthat)
  > library(performance)
  > 
  > test_check("performance")
* checking PDF version of manual ... [16s] OK
* checking HTML version of manual ... [12s] OK
* DONE
Status: 1 ERROR

(https://win-builder.r-project.org/sPqByI541sAH/)

Enabling parallel tests:

* checking tests ... ERROR
  Running 'testthat.R'
Running the tests in 'tests/testthat.R' failed.
Complete output:
  > library(testthat)
  > library(performance)
  > 
  > test_check("performance")
  Starting 2 test processes

  Error in `private$handle_error()`:
  ! testthat subprocess exited in file `test-check_collinearity.R`
  Caused by error:
  ! R session crashed with exit code -1073741819
  Backtrace:
       ▆
    1. └─testthat::test_check("performance")
    2.   └─testthat::test_dir(...)
    3.     └─testthat:::test_files(...)
    4.       └─testthat:::test_files_parallel(...)
    5.         ├─withr::with_dir(...)
    6.         │ └─base::force(code)
    7.         ├─testthat::with_reporter(...)
    8.         │ └─base::tryCatch(...)
    9.         │   └─base (local) tryCatchList(expr, classes, parentenv, handlers)
   10.         │     └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
   11.         │       └─base (local) doTryCatch(return(expr), name, parentenv, handler)
   12.         └─testthat:::parallel_event_loop_chunky(queue, reporters, ".")
   13.           └─queue$poll(Inf)
   14.             └─base::lapply(...)
   15.               └─testthat (local) FUN(X[[i]], ...)
   16.                 └─private$handle_error(msg, i)
   17.                   └─rlang::abort(...)
  Execution halted
* checking PDF version of manual ... [16s] OK
* checking HTML version of manual ... [12s] OK
* DONE
Status: 1 ERROR

(https://win-builder.r-project.org/6TYlNeXPuZ43/00check.log)

Enabling parallel tests, but skipping test-check_collinearity.R:

* checking tests ... ERROR
  Running 'testthat.R'
Running the tests in 'tests/testthat.R' failed.
Complete output:
  > library(testthat)
  > library(performance)
  > 
  > test_check("performance")
  Starting 2 test processes

  Error in `private$handle_error()`:
  ! testthat subprocess exited in file `test-check_convergence.R`
  Caused by error:
  ! R session crashed with exit code -1073741819
  Backtrace:
       ▆
    1. └─testthat::test_check("performance")
    2.   └─testthat::test_dir(...)
    3.     └─testthat:::test_files(...)
    4.       └─testthat:::test_files_parallel(...)
    5.         ├─withr::with_dir(...)
    6.         │ └─base::force(code)
    7.         ├─testthat::with_reporter(...)
    8.         │ └─base::tryCatch(...)
    9.         │   └─base (local) tryCatchList(expr, classes, parentenv, handlers)
   10.         │     └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
   11.         │       └─base (local) doTryCatch(return(expr), name, parentenv, handler)
   12.         └─testthat:::parallel_event_loop_chunky(queue, reporters, ".")
   13.           └─queue$poll(Inf)
   14.             └─base::lapply(...)
   15.               └─testthat (local) FUN(X[[i]], ...)
   16.                 └─private$handle_error(msg, i)
   17.                   └─rlang::abort(...)
  Execution halted
* checking PDF version of manual ... [16s] OK
* checking HTML version of manual ... [12s] OK
* DONE
Status: 1 ERROR

(https://win-builder.r-project.org/o3V2N1ods8J0/00check.log)

Anyone any ideas what's going on here?

strengejacke commented 3 months ago

Action for win devel is also failing: https://github.com/easystats/performance/actions/runs/8376396013/job/22936003801?pr=700

strengejacke commented 3 months ago

Furthermore, one example with glmmTMB failed (which I removed):

* checking R/sysdata.rda ... OK
* checking examples ... ERROR
Running examples in 'performance-Ex.R' failed
The error most likely occurred in:

> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: check_overdispersion
> ### Title: Check overdispersion (and underdispersion) of GL(M)M's
> ### Aliases: check_overdispersion check_overdispersion.performance_simres
> 
> ### ** Examples
> 
> ## Don't show: 
> if (getRversion() >= "4.0.0" && require("glmmTMB", quietly = TRUE)) (if (getRversion() >= "3.4") withAutoprint else force)({ # examplesIf
+ ## End(Don't show)
+ 
+ library(glmmTMB)
+ data(Salamanders)
+ m <- glm(count ~ spp + mined, family = poisson, data = Salamanders)
+ check_overdispersion(m)
+ 
+ m <- glmmTMB(
+   count ~ mined + spp + (1 | site),
+   family = poisson,
+   data = Salamanders
+ )
+ check_overdispersion(m)
+ ## Don't show: 
+ }) # examplesIf
> library(glmmTMB)
> data(Salamanders)
> m <- glm(count ~ spp + mined, family = poisson, data = Salamanders)
> check_overdispersion(m)
# Overdispersion test

       dispersion ratio =    2.946
  Pearson's Chi-Squared = 1873.710
                p-value =  < 0.001

Overdispersion detected.
> m <- glmmTMB(count ~ mined + spp + (1 | site), family = poisson, data = Salamanders)
* checking for unstated dependencies in 'tests' ... OK

I installed win-devel locally, checked the package with both glmmTMB 1.1.8 and 1.1.9 and no issues at all?

strengejacke commented 3 months ago

insight also fails for win devel: https://win-builder.r-project.org/PeZmMti2nu9v/00check.log

IndrajeetPatil commented 3 months ago

I can't reproduce this locally. The only problem I see is the following, irrespective of whether parallel testing is on or not:

Error (test-logLik.R:8:3): logLik
<expressionStackOverflowError/stackOverflowError/error/condition>
Error: evaluation nested too deeply: infinite recursion / options(expressions=)?
strengejacke commented 3 months ago

Thanks for testing! Could it be related that this code allows too many iterations/nesting?

withr::local_options(list(expressions = 25))
IndrajeetPatil commented 3 months ago

I am not sure where that option is used but that's possible.

Are we sure that there is no circular referencing between our functions in this context: performance calls effectsize which calls performance, or something of that sort?

strengejacke commented 3 months ago

I don't think so. insight is failing on win devel only, too, and this is new - I just submitted last weekend and everything was fine. On current windows, everything is ok for insight and performance, and for linux r devel, too. It must be something that recently changed, either a package update (I suspected glmmTMB because that example was failing, although it was just a simple model) or win-devel update.

strengejacke commented 3 months ago

I just saw, the loglik test is skipped on CRAN, anyway, so can't be the cause.

strengejacke commented 3 months ago

skipping all tests and examples that involve glmmTMB resolve the issues...

strengejacke commented 3 months ago

See https://github.com/glmmTMB/glmmTMB/issues/1007#issuecomment-2013810644

strengejacke commented 3 months ago

Will submit next week.