Is `check-test-warnings` broken?

[etiennebacher]

Following https://github.com/easystats/datawizard/pull/465, I removed all tests but one in datawizard and put a warning() call inside:

test_that("standardize.lm", {
  warning("foo")
  expect_equal(1+1, 2)
})

Then I ran:

options(crayon.enabled = TRUE, warn = 2L)
devtools::test()

and it showed a test warning but not a failure:

ℹ Testing datawizard
Starting 1 test process
✔ | F W  S  OK | Context
✔ |   1      1 | standardize_models                                                                                     
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Warning (test-standardize_models.R:3:3): standardize.lm
foo
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

══ Results ═════════════════════════════════════════════════════════════════════════════════════════════════════════════
Duration: 1.6 s

[ FAIL 0 | WARN 1 | SKIP 0 | PASS 1 ]

Replacing warning() by stop() correctly shows a test failure.

[IndrajeetPatil]

I can confirm that this workflow is broken. The way to make this work is to use testthat::test_dir() with options(warn = 2L). But this causes problems if parallel testing is turned on, which I suspect is a bug in {testthat}.

[IndrajeetPatil]

Bug report filed upstream https://github.com/r-lib/testthat/issues/1912

[IndrajeetPatil]

Closed in https://github.com/easystats/workflows/commit/daf6edeaabc19bc3aecaafcfdbc407b1044187f2