metrumresearchgroup / mrgvalprep

Helpers for prepping inputs to mrgvalidate
https://metrumresearchgroup.github.io/mrgvalprep
Other
1 stars 0 forks source link

fix: run_tests: don't load target package namespace #21

Closed kyleam closed 2 years ago

kyleam commented 2 years ago

run_tests(), called by validate_tests(), calls stopifnot(requireNamespace(pkg)), where pkg is the package being validated. It does this outside of the callr::r() context, which leads to packages being loaded into the current session. sessionInfo() then gets confused because it sees packages returned by loadNamespaces() that are not in the search paths returned by search().

Avoid touching the namespace by moving the stopifnot() check into the callr::r() block (which by default propagates errors to the outer session).

Fixes #2.


Here's a repeated test run of the full suite, as well as a few interspersed runs of the sys-info tests, within the same session:

devtools::test(filter = "sys-info")
devtools::test()
devtools::test(filter = "sys-info")
devtools::test()
devtools::test(filter = "sys-info")
output ``` ℹ Loading mrgvalprep ℹ Testing mrgvalprep ✔ | F W S OK | Context ✔ | 13 | get-sys-info [0.1s] ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ Duration: 0.2 s [ FAIL 0 | WARN 0 | SKIP 0 | PASS 13 ] ℹ Loading mrgvalprep ℹ Testing mrgvalprep ✔ | F W S OK | Context ✔ | 22 | end-to-end [35.4s] ✔ | 13 | get-sys-info ✔ | 2 | Optionally test hitting GHE [0.8s] ✔ | 20 | parse-github-issues [0.9s] ✔ | 35 | parse-test-output [0.4s] ✔ | 25 | read-spec-gsheet [0.8s] ✔ | 12 | Unit tests for validate-tests.R [80.4s] ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ Duration: 118.8 s [ FAIL 0 | WARN 0 | SKIP 0 | PASS 129 ] ℹ Loading mrgvalprep ℹ Testing mrgvalprep ✔ | F W S OK | Context ✔ | 13 | get-sys-info ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 13 ] ℹ Loading mrgvalprep ℹ Testing mrgvalprep ✔ | F W S OK | Context ✔ | 22 | end-to-end [26.4s] ✔ | 13 | get-sys-info ✔ | 2 | Optionally test hitting GHE [1.0s] ✔ | 20 | parse-github-issues [0.9s] ✔ | 35 | parse-test-output [0.3s] ✔ | 25 | read-spec-gsheet [0.8s] ✔ | 12 | Unit tests for validate-tests.R [74.0s] ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ Duration: 103.3 s [ FAIL 0 | WARN 0 | SKIP 0 | PASS 129 ] ℹ Loading mrgvalprep ℹ Testing mrgvalprep ✔ | F W S OK | Context ✔ | 13 | get-sys-info ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════ [ FAIL 0 | WARN 0 | SKIP 0 | PASS 13 ] ```

If I run that without the run_tests change here, I see plenty of failures like the ones mentioned in gh-2:

Warning (test-get-sys-info.R:35:3): get_sys_info() captures sessionInfo
DESCRIPTION file of package 'colorspace' is missing or broken
Backtrace:
 1. mrgvalprep::get_sys_info(session = TRUE) test-get-sys-info.R:35:2
 5. utils::sessionInfo()
 6. base::lapply(loadedOnly, packageDescription)
 7. utils:::FUN(X[[i]], ...)