Some tests can be fundamentally not amenable for parallel execution. For instance:
threads that use the capsys fixture to inspect what is written to sys.stdout and sys.stderr;
tests that use pytest.warns or more generally fiddle with the non-thread-safe warnings module of the standard library;
tests that rely on monkeypatching (for instance, via the monkeypatch fixture of unittest.mock);
...
It would be nice to have a standard way to mark such tests as unexpected to pass in a thread-based parallel execution setting, and have plugins such as pytest-freethreaded or pytest-run-parallel respect that marker to run them sequentially instead.
For convenience, the runner itself could alternatively detect known fixture usage and code pattern (usage of pytest.warns) that are fundamentally not thread-safe and automatically run those tests sequentially.
Some tests can be fundamentally not amenable for parallel execution. For instance:
capsys
fixture to inspect what is written tosys.stdout
andsys.stderr
;pytest.warns
or more generally fiddle with the non-thread-safewarnings
module of the standard library;monkeypatch
fixture ofunittest.mock
);It would be nice to have a standard way to mark such tests as unexpected to pass in a thread-based parallel execution setting, and have plugins such as
pytest-freethreaded
orpytest-run-parallel
respect that marker to run them sequentially instead.This proposal stems from this discussion on the scikit-learn issue tracker with
pytest-run-parallel
developer @andfoy.