search

pytest-dev / pytest-rerunfailures

a pytest plugin that re-runs failed tests up to -n times to eliminate flakey failures
Other
369 stars 81 forks source link

pytest_handlecrashitem faults with NotImplementedError when using the loadgroup test distribution algorithm #247

Open hb2638 opened 6 months ago

hb2638 commented 6 months ago

I'm using xdist with the --dist loadgroup command line option and the reruns are not working for crashed nodes because xdist's LoadScopeScheduling.mark_test_pending for throws a NotImplementedError error @ https://github.com/pytest-dev/pytest-xdist/blob/master/src/xdist/scheduler/loadscope.py#L244

[gw2] node down: Not properly terminated INTERNALERROR> Traceback (most recent call last): INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 271, in wrap_session INTERNALERROR> session.exitstatus = doit(config, session) or 0 INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 325, in _main INTERNALERROR> config.hook.pytest_runtestloop(session=session) INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in call INTERNALERROR> return self._hookexec(self.name, self._hookimpls, kwargs, firstresult) INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult) INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 152, in _multicall INTERNALERROR> return outcome.get_result() INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result INTERNALERROR> raise exc.with_traceback(exc.traceback) INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall INTERNALERROR> res = hook_impl.function(*args) INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/xdist/dsession.py", line 123, in pytest_runtestloop INTERNALERROR> self.loop_once() INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/xdist/dsession.py", line 148, in loop_once INTERNALERROR> call(*kwargs) INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/xdist/dsession.py", line 217, in worker_errordown INTERNALERROR> self.handle_crashitem(crashitem, node) INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/xdist/dsession.py", line 382, in handle_crashitem INTERNALERROR> self.config.hook.pytest_handlecrashitem( INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in call INTERNALERROR> return self._hookexec(self.name, self._hookimpls, kwargs, firstresult) INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult) INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 113, in _multicall INTERNALERROR> raise exception.with_traceback(exception.traceback) INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall INTERNALERROR> res = hook_impl.function(args) INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^^^ INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/pytest_rerunfailures.py", line 351, in pytest_handlecrashitem INTERNALERROR> sched.mark_test_pending(crashitem) INTERNALERROR> File "/usr/local/lib/python3.11/site-packages/xdist/scheduler/loadscope.py", line 245, in mark_test_pending INTERNALERROR> raise NotImplementedError() INTERNALERROR> NotImplementedError = 4 failed, 1155 passed, 286 skipped, 87 warnings, 10 rerun in 4137.10s (1:08:57) =

icemac commented 6 months ago

Thank you for your PR. I do not know anything about the loadgroup test distribution algorithm. So a PR is welcome to fix this issue.