pytest-dev / pytest-rerunfailures

a pytest plugin that re-runs failed tests up to -n times to eliminate flakey failures
Other
369 stars 82 forks source link

For "only_rerun" allow access exception attributes #230

Open AIGeneratedUsername opened 11 months ago

AIGeneratedUsername commented 11 months ago

I often have cases like below when I do not fail a test based on a specific exception attributes.

In case of pytest-rerunfailures the logic can look like:

@pytest.mark.flaky(
    only_rerun=ClientResponseError,
    condition=lambda error: error.status in {429, 500},
)
async def test_request():
    await request("www.example.com")

@pytest.mark.flaky(
    only_rerun=CustomDatabaseError,
    condition=lambda error: error["code"] != 123456,
)
async def test_database():
    await database.set("foo", "bar")

def _callback(error: Exception) -> bool:
    if isinstance(error, CustomDatabaseError): ...
    if isinstance(error, AnotherError): ...
    raise 

@pytest.mark.flaky(
    only_rerun=[CustomDatabaseError, AnotherError],
    condition=_callback,
)
async def test_database():
    await database.set("foo", "bar")

There are also some 3rd-party packages that raise custom exceptions and I must handle an exception in different ways based on some attributes.

Question:
Is this functionality out of scope of the current plugin and I should write a separate plugin?

icemac commented 11 months ago

@AIGeneratedUsername I think this feature request still matches the scope of this plug-in, so a PR adding a condition parameter is welcome.