search

r-lib / vdiffr

Visual regression testing and graphical diffing with testthat
https://vdiffr.r-lib.org
Other
185 stars 31 forks source link

"generated under a different vdiffr version" #144

Open r2evans opened 4 months ago

r2evans commented 4 months ago

If I test just that one test-file, then it skips and does nothing. If I test the whole package, it deletes all of the .svg files. At no time am I given the option to regenerate or even know why this behavior is occurring.

I get that there is an intention to rebuild them https://github.com/r-lib/vdiffr/blob/45bc98a9219485e51bdaf9eaa66de1f6af4ca02c/R/expect-doppelganger.R#L137-L138 but I find this behavior disrupting and likely avoidable.

Why this dead-end?

devtools::test("mypkg/", filter="950")
# ℹ Testing mypkg
# ✔ | F W  S  OK | Context
# ✔ |      1  40 | 950-plot-all-tracks [3.5s]
# ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
# Duration: 3.5 s
# ── Skipped tests (1) ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
# • SVG snapshot generated under a different vdiffr version. (1): test-950-plot-all-tracks.R:24:7
# [ FAIL 0 | WARN 0 | SKIP 1 | PASS 40 ]

devtools::test("mypkg/")
# ℹ Testing mypkg
# ✔ | F W  S  OK | Context
# ✔ |          6 | 050-approxBounded
#.....
# ✔ |      1  40 | 950-plot-all-tracks [3.0s]
# ══ Results ═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
# Duration: 9.3 s
# ── Skipped tests (1) ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
# • SVG snapshot generated under a different vdiffr version. (1): test-950-plot-all-tracks.R:24:7
# [ FAIL 0 | WARN 0 | SKIP 1 | PASS 122 ]
# Deleting unused snapshots:
# • 950-plot-all-tracks/ggplot-track-combined-1.svg
# • 950-plot-all-tracks/ggplot-track-combined-2.svg
# • 950-plot-all-tracks/ggplot-track-combined-3.svg
# • 950-plot-all-tracks/ggplot-track-combined-4.svg
#.....
# • 950-plot-all-tracks/ggplot-track-combined-31.svg
r2evans commented 4 months ago

Definitely anti-ideal workaround, at least to get to the point of comparing generated SVGs to see the differences, optionally accepting the newer versions:

$ sed -i -E "s/class='svglite'/class='svglite' data-engine-version='2.0'/"  *.svg

While the images "failed" (likely due to a pixel or two, since snapshot_review(..) revealed no visible change), all were compared, which is an improvement.

What is interesting, though, is that the resulting .svg files did not have the data-engine-version= string within them, and subsequent tests still succeeded. So perhaps there's some other way to determine different-version, or some other way that suggests a file is an "incompatible" version (whatever that means).