search

pytest-dev / pytest-rerunfailures

a pytest plugin that re-runs failed tests up to -n times to eliminate flakey failures
Other
369 stars 82 forks source link

Fix tests reported as failures when rerun happened due to exception raised from fixture teardown when using only_rerun #262

Open shlompy opened 4 months ago

shlompy commented 4 months ago

Fixes https://github.com/pytest-dev/pytest-rerunfailures/issues/261

icemac commented 4 months ago

@shlompy Thank you for your PR. Currently many tests break. Could you please have a look what's causing this?

shlompy commented 4 months ago

Thanks @icemac I hope I fixed the precommit issues. As for test_execution_count_exposed test failures, I changed the assert. It looks like the test assertion was wrong due to the bug.

I hope I have not misunderstood. This test should fail in teardown until teardown will succeed on the third run - Meaning it should have had 2 reruns and 1 pass. I'm not sure why the test asserted for 3 passes and 2 reruns in first place. Probably it took into consideration the bug, because the teardown failed twice and passed one time, but when it failed twice, it was too late because the 'call' (test) report was already concluded as pass.

Now with my bug fix the outcome is indeed 1 pass and 2 reruns, so I've updated the test assertion to match that.

But I don't see the test git actions being triggered again after my last push

hugovk commented 4 months ago

Thanks for the review request, but I'll stick to general CI/maintenance things for this library and leave the internals to others 👍

shlompy commented 4 months ago

resolved conflicts

shlompy commented 4 months ago

gentle reminder, can this fix please be reviewed?