Make assertion logging for compairsons more helpful?

[bandophahita]

This may not even be possible at the screenpy level and perhaps it's something we pass along to hamcrest.....

It sure would be nice if the failed assert comparing two strings would line up the strings to make it easier to identify where the difference lies.

Here is a current example of two large strings that do not match.

AssertionError: 
Expected: a string containing 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\nJob Category to be edited\nEdit\nACTIVE'
     but: was 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\neditable category\nEdit\nACTIVE'

It'd be nice if it looked something more like this:

AssertionError: 
Expected: 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\nJob Category to be edited\nEdit\nACTIVE'
     got: 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\neditable category\nEdit\nACTIVE'
          ^ line up here

[perrygoy]

Ironically, the "current example" was easier for me to find the error in because the substring was closer to the almost-matching substring in the non-matching string. But i do get what you're getting at.

Removing the "a string containing" part of the failed assertion message will lose valuable information, though. In your changed example, i would assume we were looking for an exact match.

You're right, we'd probably have to make that change in Hamcrest, or make custom versions of all of the matchers we want this to affect. Then we could sub in the changed mismatch string, maybe just have the objects under test on their own lines?

AssertionError: 
Expected: a string containing 
        'Category Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\nJob Category to be edited\nEdit\nACTIVE'
     but: was 
         'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\neditable category\nEdit\nACTIVE'

[bandophahita]

I updated the example to illustrate a little better.

[bandophahita]

Or something like this:

AssertionError: 
Expected: a string containing 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\nJob Category to be edited\nEdit\nACTIVE'
     but: was actually this   'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\neditable category\nEdit\nACTIVE'
                              ^ line up here

I would guess two lines might be a better solution though.

[perrygoy]

Maybe for strings we could make something even more tester-friendly and use ndiff?

That could produce output like this:

AssertionError:
    Expected: a string containing 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\nJob Category to be edited\nEdit\nACTIVE'
     but: was 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\neditable category\nEdit\nACTIVE'
         which has: 'editable category'
         instead of: 'Job Category to be edited'

This is what i did:

>>> ndiff('editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\nJob Category to be edited\nEdit\nACTIVE'.splitlines(keepends=True), 'editable category\nCategory Code: EDITCAT\nClass Type:\nClass Code:\nDescription:\nIndirect\nEDITCLASS\neditable category\nEdit\nACTIVE'.splitlines(keepends=True))
<generator object Differ.compare at 0x105269f80>
>>> [x for x in _]
['  editable category\n', '  Category Code: EDITCAT\n', '  Class Type:\n', '  Class Code:\n', '  Description:\n', '  Indirect\n', '  EDITCLASS\n', '- Job Category to be edited\n', '+ editable category\n', '  Edit\n', '  ACTIVE']

We could then check which entries in the diff results start with + and - and use that information to fill out the "which had" and "instead of" lines, respectively.

[bandophahita]

Using diff tools can be really tricky depending on the differences especially when it comes to patterns etc. I started to suggest we opt for displaying the visual differences to avoid possible confusion, but I got to wondering if this is something that should be able to customize to their need?

[perrygoy]

Users can already customize it to their need by doing the exact thing that i'm suggesting we do, haha. :P

[bandophahita]

I submitted a ticket to hamcrest: https://github.com/hamcrest/PyHamcrest/issues/235

[perrygoy]

Since we've got the issue open in PyHamcrest, i'll close this one here. They have it marked as a "good first issue," maybe i'll take it on or something!