ScreenPyHQ / screenpy

Screenplay pattern base for Python automated UI test suites.
MIT License
28 stars 4 forks source link

Eventually logging multiple occurances of the same exception #102

Closed bandophahita closed 1 year ago

bandophahita commented 1 year ago

Eventually doesn't always recognize multiple occurrences of exceptions as being the same:

DeliveryError: Marcel tried to Eventually see if side nav expanded is equal to <True> 37 times over 10 seconds, but got:
    AssertionError: 
Expected: <True>
     but: was <False>

    AssertionError: 
Expected: <True>
     but: was <False>

    AssertionError: 
Expected: <True>
     but: was <False>

    AssertionError: 
Expected: <True>
     but: was <False>

    AssertionError: 
Expected: <True>
     but: was <False>

    AssertionError: 
Expected: <True>
     but: was <False>

    AssertionError: 
Expected: <True>
     but: was <False>

    AssertionError: 
Expected: <True>
     but: was <False>
bandophahita commented 1 year ago

The problem appears to be that for each exception that is raised, the arguments passed in can have their own unique object IDs. In this case hamcrest creates StringDescription object that contains the same string value, but each object has a different object ID.

image
>>> a == b
False
>>> type(a) == type(b)
True
>>> a.args == b.args
False
>>> a.args
(<hamcrest.core.string_description.StringDescription object at 0x1099b2fd0>,)
>>> b.args
(<hamcrest.core.string_description.StringDescription object at 0x1099af9d0>,)
>>> a.args[0].out == b.args[0].out
True
>>> str(a) == str(b)
True
perrygoy commented 1 year ago

Fascinating, that makes sense because they do that complicated Description building to make their messages. Even if the string ends up being the same, the objects will be different.

I think what we landed on in Slack, using f"{exc.__class__}{str(exc)}" or something like that for the key would be the way to go.

bandophahita commented 1 year ago

It's unclear to me if the str(exc) was enough to compare "like" exceptions. So I created a function that does a couple of checks.