mr-sun / hamcrest

Hamcrest matchers for c++ (using with carnamock)
0 stars 0 forks source link

Show a readable diff as part of the failure description for assertThat #213

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
For a simple check like:

    assertThat(manyLinesOfText, is(equalTo(expectedOutput)));

The resulting AssertionError message may well have only one character 
different, but it shows both full strings anyway. Usually this forces us to 
paste the strings into a text editor, unescape the content, save two files and 
diff the files. So basically, this wastes a lot of user time every time a test 
fails.

Now... IDEs could show a diff, and many try, but all they get from the test 
result is the text of an AssertionError, so they have to use regular 
expressions to match the messages to figure out which ones are equality 
comparisons. IDEA, for instance, appears to be taking years to get this regex 
right and I filed yet another bug against it today for another case which it 
doesn't match.

Build systems such as Jenkins do have plugins available which are supposed to 
perform a similar task, but they are all having similar problems and none of 
the ones we have tried so far will show diffs for any failures at all.

So since this is apparently too hard for the caller to solve, I think it does 
make sense to fix it at the location where the message is generated.

This is not without precedent. With rspec, for example, if I have an example 
which compares multiple lines of text:

    describe 'the thing' do
      it 'does something' do
        expect("Line 1\nLine 2\n").to eq("Line 1\nLine 3\n")
      end
    end

then I do get a diff in the failure:

      1) the thing does something
         Failure/Error: expect("Line 1\nLine 2\n").to eq("Line 1\nLine 3\n")

           expected: "Line 1\nLine 3\n"
                got: "Line 1\nLine 2\n"

           (compared using ==)

           Diff:
           @@ -1,3 +1,3 @@
            Line 1
           -Line 3
           +Line 2
         # ./test/test.spec:3:in `block (2 levels) in <top (required)>'

It does this for comparing hashes and arrays as well, which is helpful in the 
same sort of way.

If this could happen for Hamcrest's assertion failures, that would be extremely 
helpful.

Original issue reported on code.google.com by trejkaz on 21 Jul 2015 at 6:44