`pytest-rerunfailures` slows down `pytest-xdist` testing

pytest-dev / pytest-rerunfailures

a pytest plugin that re-runs failed tests up to -n times to eliminate flakey failures

Other

369 stars 82 forks source link

`pytest-rerunfailures` slows down `pytest-xdist` testing #242

Closed mtelka closed 2 hours ago

mtelka commented 8 months ago

I tried to run tests for pytest-xdist with pytest-rerunfailures installed and disabled using -p no:rerunfailures set in PYTEST_ADDOPTS environment variable. I noticed that with this setup the pytest-xdist tests takes very long time (over one hour on my testing machine) and CPU is heavily utilized. Without the pytest-rerunfailures installed the pytest-xdist testing completes in about 3 minutes on my machine.

Version info:

pytest-rerunfailures 12.0
pytest-xdist 3.4.0
pytest 7.4.3
Python 3.9.16

mtelka commented 8 months ago

Actually, the testing takes far more than one hour because it looks like it exponentially slows down. After ca 20 minutes the pytest-xdist testing progress was at about 30%, after 50 minutes it was at 40%. After that I had to stop the test run.

icemac commented 8 months ago

Support for xdist was added in #53 (6 years ago) and improved in several following PRs. There have not yet been any issues reported. Thus I have no idea what could be the problem there.

mtelka commented 2 hours ago

Since the pytest-rerunfailures is not needed to test pytest-xdist then we should simply do not install (or load) it while testing pytest-xdist.