r-lib / testthat

An R 📦 to make testing 😀
https://testthat.r-lib.org
Other
892 stars 319 forks source link

Show values of expected and actual values for comparisons #104

Closed krlmlr closed 10 years ago

krlmlr commented 10 years ago

I find it difficult to determine, for a failing expect_equal expectation, the actual value that has been computed. It would be great if, in this case, the expected and actual values would be printed. Is there an option to enable this?

If not: Perhaps a good place to implement this is a wrapper around all.equal, to be called by compare.default, that will return actual values in certain cases but otherwise call all.equal.

stephenll commented 10 years ago

I also would like the ability to have a flag or part of the summary report the actual and expected values. I am often comparing matrices and this would help greatly. I started to look through the code to find out how I can do this myself but I have not made any progress yet. I am a new R user coming from matlab.

FYI I have seen this in practice in Visual Studio 2013 and it was very handy to have.

Thanks for a great package.

hadley commented 10 years ago

This is available in the development version. See the compare generic and its methods for details.

It's rather difficult to do this in a completely generic way because you don't want to accidentally print huge amounts of data to the console. Typically, I rely on the tests to identify that there is an error, then use browser() etc to figure out exactly what's wrong.

dplyr also has a nicer implementation of all.data.frame that can ignore row and column order, and provides more informative error messages.

krlmlr commented 10 years ago

With most recent testthat from GitHub, I'm seeing the following:

a <- 3; b <- 5; testthat::expect_equal(a, b)
## Error: a not equal to b
## Mean relative difference: 0.4

I'd expect to see something like

## Error: a not equal to b
## 1 number mismatches:
## x[1]: 5
## y[1]: 3

Just like this is implemented for strings, see http://rpubs.com/krlmlr/testthat-104 .

hadley commented 10 years ago

Sorry, I meant it was available for strings, but you could provide other methods if you want. But figuring out how those methods should work is hard - it's easy if you have short vectors, but it also needs to work gracefully for long vectors. You also need to think about special cases like the vectors are different types, or different lengths, or contain the same values but in different orders. See https://github.com/hadley/dplyr/issues/110 for more discussion.

krlmlr commented 10 years ago

But compare.string already offers much of the logic needed. The only thing required would be a substitution for x != y in line 34 of compare.r that also works for other types (e.g., numerics are compared with a tolerance by default) and does not overflow with data frames (perhaps we can borrow from dplyr here).

hadley commented 10 years ago

I'd be happy to take a pull request, but I don't currently have the time to think this through myself.

wheelsandmetal commented 6 years ago

Is there a way to do this now?

krlmlr commented 6 years ago

Commenting to a issue from four years ago isn't really helpful. For questions, please visit https://community.rstudio.com -- if this concerns code, a reproducible example helps a lot: https://reprex.tidyverse.org/.