search

r-lib / testthat

An R 📦 to make testing 😀
https://testthat.r-lib.org
Other
888 stars 318 forks source link

Snapshot differences on Mac/Windows/Ubuntu #1934

Closed nikosbosse closed 7 months ago

nikosbosse commented 7 months ago

Recently (~3days ago) we saw failing snapshot tests for our package scoringutils. CI checks that previously passed were failing on macOS-latest (release). As of today, they are also failing on windows-latest (release).

This is the current gh actions run on main: https://github.com/epiforecasts/scoringutils/actions/runs/8023799140.

After the test failure, I updated all packages locally and updated the snapshots (PR). CI checks are passing on macOS-latest (release), windows-latest (release), and ubuntu-latest (3.6) and are failing on ubuntu-latest (devel), ubuntu-latest (release) (pull_request) ubuntu-latest (oldrel-1).

This is the github action yaml file.

Since testthat hasn't recently changed on CRAN I don't think it is directly related to the package. But I wanted to flag this just in case it is useful to you and in case anyone has an idea of what to do. Is there any other information I could provide that would be helpful? Thank you very much!

Local sessionInfo: 

R version 4.3.2 (2023-10-31)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Sonoma 14.2.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/Mexico_City
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] shiny_1.8.0             scoringutils_1.2.2.9000

loaded via a namespace (and not attached):
 [1] ggdist_3.3.1         sass_0.4.8           utf8_1.2.4           generics_0.1.3       digest_0.6.34       
 [6] magrittr_2.0.3       grid_4.3.2           scoringRules_1.1.1   fastmap_1.1.1        jsonlite_1.8.8      
[11] backports_1.4.1      brio_1.1.4           promises_1.2.1       fansi_1.0.6          scales_1.3.0        
[16] jquerylib_0.1.4      cli_3.6.2            rlang_1.1.3          ellipsis_0.3.2       munsell_0.5.0       
[21] yaml_2.3.8           cachem_1.0.8         diffviewer_0.1.1     tools_4.3.2          Metrics_0.1.4       
[26] memoise_2.0.1        checkmate_2.3.1      dplyr_1.1.4          colorspace_2.1-0     ggplot2_3.5.0       
[31] httpuv_1.6.14        vctrs_0.6.5          R6_2.5.1             mime_0.12            lifecycle_1.0.4     
[36] htmlwidgets_1.6.4    MASS_7.3-60.0.1      pkgconfig_2.0.3      pillar_1.9.0         bslib_0.6.1         
[41] later_1.3.2          gtable_0.3.4         data.table_1.15.0    glue_1.7.0           Rcpp_1.0.12         
[46] xfun_0.42            tibble_3.2.1         tidyselect_1.2.0     rstudioapi_0.15.0    knitr_1.45          
[51] xtable_1.8-4         htmltools_0.5.7      testthat_3.2.1       compiler_4.3.2       distributional_0.4.0
nikosbosse commented 7 months ago

It seems that the issue was caused by an update to ggplot2 on February 23, 2024 on CRAN. I suspect that some checks are failing (and not others) due to the fact that it takes some time to update the package for all systems. I expect this to be resolved over the next few days and am going to close the issue for now.