Open icemac opened 6 months ago
This is now affecting pytest 8.2.0.
This looks like a repeat of what https://github.com/pytest-dev/pytest-rerunfailures/commit/0ab54f00d68bb4feadae58d2f4e30f6663495571 tried to address (the teardown fixtures are not being called correctly). I spent some time debugging this this afternoon and the logic still looks correct, although it is effectively the same error. I'm not too familiar with the pytest internals but I wonder if the fix is somewhere along the lines of keeping a reference to the original setupstate instead of the depending on the final tests setupstate object?
Hi guys, I think we've faced the same problem using pytest-rerunfailures with pytest 8.2.2 and pytest-xdist 3.6.1. we run tests with xdist loadscope and rerunfailures and on the step of collecting tests it fails:
platform linux
pytest-8.2.2
configfile: pytest.ini
plugins: rerunfailures-14.0, xdist-3.6.1
created: 3/3 workers
3 workers [43 items]
scheduling tests via LoadScopeScheduling
INTERNALERROR> def worker_internal_error(
INTERNALERROR> self, node: WorkerController, formatted_error: str
INTERNALERROR> ) -> None:
INTERNALERROR> """
INTERNALERROR> pytest_internalerror() was called on the worker.
INTERNALERROR>
INTERNALERROR> pytest_internalerror() arguments are an excinfo and an excrepr, which can't
INTERNALERROR> be serialized, so we go with a poor man's solution of raising an exception
INTERNALERROR> here ourselves using the formatted message.
INTERNALERROR> """
INTERNALERROR> self._active_nodes.remove(node)
INTERNALERROR> try:
INTERNALERROR> > assert False, formatted_error
We had to rollback to pytest 7.4.4 but left rerunfailures-14.0, xdist-3.6.1 and now it works properly. Could you look at it please thanks
With pytest==8.2.2 and pytest-xdist==3.6.1, and pytest-rerunfailures==14.0, all tests that undergo a rerun result in the following AssertionError:
def runtest(self) -> None:
from _pytest.debugging import maybe_wrap_pytest_function_for_tracing
testcase = self.instance
> assert testcase is not None
E AssertionError
It works fine for me on pytest==8.2.1 and pytest-cdist==3.5.0 and pytest-rerunfailures==14.0.
It seems reruns are not working with pytest==8.2.2 and pytest-rerunfailures==14.0, anyone aware of any workarounds?
The currently only known workaround is to use an older version of pytest
.
@asottile-sentry had teardown issues and bisected it to https://github.com/pytest-dev/pytest/pull/11833 (which I authored). https://github.com/pytest-dev/pytest/issues/12135 could also be the cause of some of the errors.
I'm not familiar with how pytest-rerunfailures messes with teardowns, but maybe looking through those PRs can help you debug.
Thanks @jakkdl I can confirm that pytest-dev/pytest@70c11582aaed is the exact commit that causes the fail of five tests in pytest-rerunfailures test suite:
test_run_session_teardown_once_after_reruns
test_exception_matches_rerun_except_query
test_exception_not_match_rerun_except_query
test_exception_matches_only_rerun_query
test_exception_match_only_rerun_in_dual_query
See https://github.com/pytest-dev/pytest-rerunfailures/actions/runs/8409152908/job/23026172136
This
can be a temporary problem or ais a hint that we have to change something here to make it compatible again.