brodieG / unitizer

Easy R Unit Tests
Other
39 stars 6 forks source link

Tests fail with testthat 3.0.0 #270

Closed QuLogic closed 3 years ago

QuLogic commented 3 years ago

When running with testthat 2.3.2, tests pass, but with 3.0.0, they fail:

--------------------------------------------------------------------------------
FAILURE (testthat.utz1.R:279:5): namespace conflict
`txt4` has changed from known value recorded in 'helper/refobjs/unitize_nsconf1.rds'.
Component "message": 1 string mismatch

FAILURE (testthat.utz1.R:282:5): namespace conflict
`txt5` has changed from known value recorded in 'helper/refobjs/unitize_nsconf2.rds'.
Component "message": 1 string mismatch

FAILURE (testthat.utz1.R:288:5): namespace conflict
`txt7` has changed from known value recorded in 'helper/refobjs/unitize_nsconf3.rds'.
Component "message": 1 string mismatch
brodieG commented 3 years ago

Will you please provide sessionInfo()? This works for me with 3.0. One possible issue could be that you don't have covr available which is required for those tests to work properly, but I don't know why that would work with 2.3 but not 3.0.

Also, out of curiosity, why are you running the unitizer tests?

brodieG commented 3 years ago

Also, FWIW I just pushed the just-released 1.4.11 to master, not sure what version you were running.

brodieG commented 3 years ago

@QuLogic turns out I need to resubmit to CRAN due to failing old-rel tests, so if indeed there is something wrong here it would be a good chance to fix it. Will you post your sessionInfo() that produced the errors?

QuLogic commented 3 years ago

Sorry for the delay, here is the sessionInfo():

+ Rscript -e 'sessionInfo()'  
R version 4.0.3 (2020-10-10)   
Platform: x86_64-redhat-linux-gnu (64-bit)          
Running under: Fedora 34 (Rawhide Prerelease)
Matrix products: default
BLAS/LAPACK: /usr/lib64/libflexiblas.so.3.0
locale:
[1] C
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     
loaded via a namespace (and not attached):
[1] compiler_4.0.3

I'm packaging for Fedora, so I try to always run tests to ensure everything is working.

brodieG commented 3 years ago

Hmm, I can't reproduce. My guess is you don't have covr available (or did not at the time you ran the tests), but that the last time you ran with testthat v2 you did have covr. If I remove.packages('covr') and restart session / re-run tests I get the same errors as you do (I know this because I forgot covr was required for tests and tried to remove it from the "suggests" dependency list as part of this release, and the exact same tests failed). The package works fine, it's just that those tests specifically behave differently if covr is or isn't available due to some covr behavior. covr is a "suggested" unitizer package, so it is needed for tests.

Thanks for taking the time to report this, especially since you probably run into a lot of these as part of the packaging work. If you know that covr was installed at the time the test was run, let me know.

QuLogic commented 3 years ago

Ah, you're right; I normally leave out covr because 99% of the time it's there for CI, development, and such things that upstream wants to track, but we couldn't possibly do anything about downstream. But it makes sense that unitizer, as a testing-related package, might need it and tests do work once it's installed. I guess it was a red herring that testthat 2->3 caused the failure.