r-lib / testthat

An R 📦 to make testing 😀
https://testthat.r-lib.org
Other
883 stars 317 forks source link

Prevent snapshot cleanup #1481

Open rickhelmus opened 2 years ago

rickhelmus commented 2 years ago

Hello,

I have quite a lot of tests relying on vdiffr, which recently started using snapshots. Which tests are executed is pre-determined by a shell environment variable. However, this conditional system doesn't play nicely anymore with the automatic cleaning of 'unused' snapshot files, as all the reference snapshots from tests not being executed are removed. I wondered if there is a way to prevent activation of the automatic cleaning system?

Thanks, Rick

hadley commented 2 years ago

Do you have an example you could point me to? I think you might be able to use variants to help with this.

hadley commented 2 years ago

Ok, the root problem is coming back to me — when you run expect_snapshot_file(..., name = "foo"), testthat registers foo as a known file, and something that shouldn't be deleted. If expect_snapshot_file() isn't run (regardless of how), that name never gets registered, and the file is slated for automatic deletion. One way to work around this problem is to use annouce_file_snapshot() which registers the name without running the test. This looks something like:

if (run_snaps()) {
  expect_snapshot_file(path, name = "foo")
} else {
  announce_file_snapshot(name = "foo")
  skip("Snaps off")
}

This is obviously a bit clunky, but you could wrap up into a helper:

my_expect_snapshot_file <- function(path, name = basename(path), ...) {
    if (run_snaps()) {
    expect_snapshot_file(path, name = name, ...)
  } else {
    announce_file_snapshot(name = name)
    skip("Snaps off")
  }
}

It's possible that testthat could do more here, but I'm not sure what, and hopefully this code helps a little.