New RNG version in R-devel (R 3.6.0) breaks package

HenrikBengtsson commented 5 years ago

On R-devel (to become R 3.6.0) we're now getting:

--- re-building ‘c3co.Rmd’ using rmarkdown
W is rank deficient: there are too many archetypes
Quitting from lines 113-121 (c3co.Rmd) 
Error: processing vignette 'c3co.Rmd' failed with diagnostics:
missing value where TRUE/FALSE needed
--- failed re-building ‘c3co.Rmd’

For example: Travis CI failure https://travis-ci.org/pneuvial/c3co/jobs/500988047

Also, if building package without vignettes and then running R CMD check, we get:

* checking tests ...
  Running ‘testthat.R’
 ERROR
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
  max((Zhat[[mm]] - Zt[[mm]])^2) is not strictly less than `tol`. Difference: 0.000485

  ── 5. Failure: positiveFusedLasso recovers the truth in (almost) noiseless situa
  max((Zhat[[mm]] - Zt[[mm]])^2) is not strictly less than `tol`. Difference: 0.000488

  ══ testthat results  ═══════════════════════════════════════════════════════════
  OK: 644 SKIPPED: 3 FAILED: 5
  1. Failure: c3co terminates on TCN (@test_c3co.R#54) 
  2. Failure: get.Z recovers Z in (almost) noiseless situations for small lambda (@test_get.Z.R#43) 
  3. Failure: get.Z recovers Z in (almost) noiseless situations for small lambda (@test_get.Z.R#43) 
  4. Failure: positiveFusedLasso recovers the truth in (almost) noiseless situations for small lambda (@test_positiveFusedLasso.R#52) 
  5. Failure: positiveFusedLasso recovers the truth in (almost) noiseless situations for small lambda (@test_positiveFusedLasso.R#52) 

  Error: testthat unit tests failed
  Execution halted

HenrikBengtsson commented 5 years ago

Test details:

> testthat::test_file("tests/testthat/test_c3co.R")
✔ | OK F W S | Context
✖ |  0 1 3   | c3co test [2.9 s]
────────────────────────────────────────────────────────────────────────────────────────────────────
test_c3co.R:41: warning: c3co terminates on C1,C2
For model with 3 features, some components in minor latent profiles are larger than matched components in major latent profiles

test_c3co.R:41: warning: c3co terminates on C1,C2
For model with 4 features, some components in minor latent profiles are larger than matched components in major latent profiles

test_c3co.R:41: warning: c3co terminates on C1,C2
For model with 5 features, some components in minor latent profiles are larger than matched components in major latent profiles

test_c3co.R:54: failure: c3co terminates on TCN
`c3co(dat, parameters.grid = parameters.grid, stat = "TCN")` did not throw an error.
────────────────────────────────────────────────────────────────────────────────────────────────────

> testthat::test_file("tests/testthat/test_get.Zt.R")
✔ | OK F W S | Context
✖ | 100 2     | get.Z [0.6 s]
────────────────────────────────────────────────────────────────────────────────────────────────────
test_get.Zt.R:43: failure: get.Z recovers Z in (almost) noiseless situations for small lambda
max((Z - Z1hat)^2) is not strictly less than `tol`. Difference: 0.156

test_get.Zt.R:43: failure: get.Z recovers Z in (almost) noiseless situations for small lambda
max((Z - Z1hat)^2) is not strictly less than `tol`. Difference: 0.154
────────────────────────────────────────────────────────────────────────────────────────────────────

> testthat::test_file("tests/testthat/test_positiveFusedLasso.R")
✔ | OK F W S | Context
✖ | 282 2     | positiveFusedLasso [1.1 s]
────────────────────────────────────────────────────────────────────────────────────────────────────
test_positiveFusedLasso.R:52: failure: positiveFusedLasso recovers the truth in (almost) noiseless situations for small lambda
max((Zhat[[mm]] - Zt[[mm]])^2) is not strictly less than `tol`. Difference: 0.000485

test_positiveFusedLasso.R:52: failure: positiveFusedLasso recovers the truth in (almost) noiseless situations for small lambda
max((Zhat[[mm]] - Zt[[mm]])^2) is not strictly less than `tol`. Difference: 0.000488
────────────────────────────────────────────────────────────────────────────────────────────────────

> sessionInfo()
R Under development (unstable) (2019-03-01 r76186)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /home/hb/software/R-devel/trunk/lib/R/lib/libRblas.so
LAPACK: /home/hb/software/R-devel/trunk/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] c3co_0.4.0          Biobase_2.43.1      BiocGenerics_0.29.1 testthat_2.0.1     

loaded via a namespace (and not attached):
 [1] modeltools_0.2-22  tidyselect_0.2.5   lpSolve_5.6.13     purrr_0.3.0        reshape2_1.4.3    
 [6] lattice_0.20-38    limSolve_1.5.5.3   colorspace_1.4-0   stats4_3.6.0       NMF_0.21.0        
[11] rlang_0.3.1        pillar_1.3.1       glue_1.3.0         withr_2.1.2        RColorBrewer_1.1-2
[16] registry_0.5       rngtools_1.3.1     matrixStats_0.54.0 foreach_1.4.4      plyr_1.8.4        
[21] pkgmaker_0.27      stringr_1.4.0      archetypes_2.2-0   munsell_0.5.0      gtable_0.2.0      
[26] codetools_0.2-16   doParallel_1.0.14  Rcpp_1.0.0         xtable_1.8-3       scales_1.0.0      
[31] ggplot2_3.1.0      digest_0.6.18      stringi_1.3.1      dplyr_0.8.0.1      jointseg_1.0.2    
[36] grid_3.6.0         bibtex_0.4.2       quadprog_1.5-5     cli_1.0.1          tools_3.6.0       
[41] magrittr_1.5       lazyeval_0.2.1     glmnet_2.0-16      tibble_2.0.1       cluster_2.0.7-1   
[46] crayon_1.3.4       pkgconfig_2.0.2    MASS_7.3-51.1      Matrix_1.2-15      gridBase_0.4-7    
[51] acnr_1.0.0         nnls_1.4           assertthat_0.2.0   iterators_1.0.10   R6_2.4.0          
[56] compiler_3.6.0

HenrikBengtsson commented 5 years ago

This bug has to do with the following from R 3.6.0 NEWS:

The default method for generating from a discrete uniform distribution (used in sample(), for instance) has been changed. This addresses the fact, pointed out by Ottoboni and Stark, that the previous method made sample() noticeably non-uniform on large populations. See PR#17494 for a discussion. The previous method can be requested using RNGkind() or RNGversion() if necessary for reproduction of old results. Thanks to Duncan Murdoch for contributing the patch and Gabe Becker for further assistance.

The errors do not occur if we use the old RNG version:

export _R_RNG_VERSION_=3.5.2

HenrikBengtsson commented 5 years ago

Fixed by commit 0526d16, but see also Issue #76 because we might produce "garbage" output.

pneuvial / c3co

New RNG version in R-devel (R 3.6.0) breaks package #73