Closed dingyunxing closed 5 years ago
You have to call vdiffr::manage_cases()
(which has an RStudio addin) on the package where you have added the visual tests.
@lionel- yeah, but after i call the manage_cases()
, I only got a shiny page without any plot like what you post in "readme" but just told me test done. There is nothing on toggle, slide or diff. Honestly I am not very clear about how this package works, and I found the vdiffr::manage_cases()
will not only test the graph but will also run all the test case in testthat.
My output is just like this:
vdiffr::manage_cases() will not only test the graph but will also run all the test case in testthat.
That is expected. Did you put the vdiffr cases in a testthat file? If you didn't that is why it couldn't find any.
Honestly I am not very clear about how this package works
This package kinda works but is still experimental. We need to make important internal changes to make it reliable on third party machines like CRAN checking servers. Once this is done we'll work on the docs and advertise the package to the public.
Sorry, @lionel- I have tried hard but still failed. Where should I put the vdiffr cases? Is the case mean a .svg graph or something else? Could you give me an example of
vdiffr::expect_doppelganger("Base graphics histogram", disp_hist_base) vdiffr::expect_doppelganger("Rplot", disp_hist_ggplot)
you posted in readme?
In the testthat, what i usually do is writing code like:
expect_equal(disp_hist_ggplot, hist(mtcars$disp))
the disp_hist_ggplot
is what i want to test while the hist(mtcars$disp)
is what i expected.
I just tried to put a .svg graph (which is what i expected to test) to the tests/figs/, but it only collected one Orphaned case.
This package is really helpful to my current project so many thanks to you!
You should not create svg files yourself. The steps are to add expect_doppelganger()
statements in test_that()
blocks in a testthat file of unit tests and and then to call manage_cases()
. From there you will be able to validate new cases (which will be placed the svg files in the figs folder).
This is the code in a file "test-ggplot.R" in dir: ".../packagename/tests/testthat/ "
test_that("test ggplot", {
disp_hist_base <- function() hist(mtcars$disp)
disp_hist_ggplot <- ggplot(mtcars, aes(disp)) + geom_histogram()
vdiffr::expect_doppelganger("Base graphics histogram", disp_hist_base)
vdiffr::expect_doppelganger("ggplot2 histogram", disp_hist_ggplot)
})
Then I ran the manage_cases()
but still nothing showed in the shiny.
Are you using an RStudio project? If not is your working directory the package root?
Yeah, I am working on a RStudio project and it is a private package works all well. I just add a new function in it to do the plotting so I also want to add a test case in it. The working directory is this package root because when I run the code
vdiffr::expect_doppelganger("Base graphics histogram", disp_hist_base)
vdiffr::expect_doppelganger("ggplot2 histogram", disp_hist_ggplot)
I can clearly see there is a new folder "figs" created in the dir "package/tests/", but nothing inside.
Sorry about that, could you please run collect_cases()
and paste the full output here.
Oh, it works!!! When I change the working dir to "packagename/", it works now! but the folder figs created not in the packagename/tests/, but in the same level of the package(with the same parent dir), I don't know why .. I think it should be sitted in the dir "packagename/tests/figs" By the way, thank you very much!!! @lionel-
I think we need some kind of package root detection mechanism, e.g. based on the presence of a DESCRIPTION
file somewhere in the parent folders.
the folder figs created not in the packagename/tests/, but in the same level of the package
Weird, the figures should indeed be created in pkgroot/tests/figs/
.
Yeah, in my case, I put my package at the Desktop. when I run the code:
vdiffr::expect_doppelganger("Base graphics histogram", disp_hist_base)
vdiffr::expect_doppelganger("ggplot2 histogram", disp_hist_ggplot)
I found the "figs" folder was created in the Desktop.
Then I run the manage_cases()
, I found another "figs" folder created in the pkgroot/tests/figs/
I have deleted both of them and tried again, same thing happened.
expect_doppelganger()
is not meant to be run interactively. I think it should return an error if you try to do so. Does Desktop/figs
gets created even if you never call expect_doppelganger()
interactively?
Oh, yes! it won't appear when I only run the manage_case()
Why the "Diff" page is all black for the base graphic histogram while all white for the ggplot2 histogram?
I don't know but the diff doesn't matter if that's a new testcase.
Cool! In terms of the new case, I found you also created the mismatched case in your example, how did you do it? Did you create another test case?
@lionel- Excuse me, I can only validate the case but failed to mark it as failed so I cannot call the special reporter with collect_case
to generate the mismatched one. I also checked the source code of the special reporter but didn't find how to call the stuff like "mismatch", "match", "failure", etc.
just make any change to the plot that was validated and rerun the case manager, you should get a failure.
@lionel- Sorry, maybe the last question:
I tried a lot of different changes to the validated plot, the easiest one may change the hist()
to par()
.
But always the same error occurs:
Error: while collecting vdiffr cases. Last error:
test: test ggplot
message: object of type 'closure' is not subsettable
Sorry, I change the hist()
to pie()
, not par()
This looks like the same issue as on https://cran.r-project.org/web/checks/check_results_vdiffr.html
This is a bug that needs fixing.
What a pity……But still thanks a lot!
I just run the example code: context("Histograms")
disp_hist_base <- function() hist(mtcars$disp) disp_hist_ggplot <- ggplot(mtcars, aes(disp)) + geom_histogram()
vdiffr::expect_doppelganger("Base graphics histogram", disp_hist_base) vdiffr::expect_doppelganger("ggplot2 histogram", disp_hist_ggplot)
but it only generate a vacant folder called "figs/histogram" but there is no svg file in it. When I try to use vdiffr::expect_doppelganger("ggplot2 histogram", disp_hist_base, verbose = T) it always said: "Failed doppelganger: base-graphics-histogram"