Open A5rocks opened 1 month ago
Hi, I want try to explain the difference in outcomes between the two execution methods:
The warning is captured during the collection and execution of test cases after pytest is initialized and ready.
If you execute
pytest test.py
pytest will import example
during the collection phase, and the DeprecationWarning
will be captured.
If you execute
pytest test.py -p example
pytest will first import the example
plugin during the initialization phase. At this point, the DeprecationWarning
is raised, but according to Python's default rules, the warning is ignored (reference: https://docs.python.org/3/library/exceptions.html#DeprecationWarning).
Later, pytest will import example
again during the collection phase, but this time, the DeprecationWarning
won't be raised again, so it cannot be captured.
The root cause is that Python ignores the DeprecationWarning
, not pytest.
If you want to display warnings regardless, you can:
DeprecationWarning
to a type that is not ignored by Python, such as UserWarning
, PytestDeprecationWarning
, etc.DeprecationWarning
.If you want to ensure that warnings are not displayed under any circumstances, you can:
pytest.ini
to instruct pytest to ignore DeprecationWarning
.I'm aware of why this currently doesn't work, but it isn't immediately obvious and has caused (at least) one case where tests didn't match reality.
Specifically, shouldn't it be possible for pytest to capture warnings from plugin imports and apply its warning filter on them? That isn't status quo, yes, but I think that would be a less surprising place for pytest to be. (Though maybe trio's case of using a plugin instead of conftest.py for privacy reasons is unique? And so this isn't worth it?)
Right, Shouldn't Python, pytest, and plugins all capture warnings from plugin imports and apply their warning filter on them?
If Python, as the upstream, and the plugins, as the downstream, didn't do this, it is debatable for pytest to do so on its own. From the perspective of a plugin maintainer, I believe that changing the upstream software to alter the default behavior of the further upstream should be handled with caution
pytest should enact its warning filters before starting to import from entrypoitns/given plugins of possible
When importing a package for its plugin, pytest lets any warnings fall to the ground. This means that subsequent imports in tests do not get any warnings, meaning it's possible to miss deprecation warnings. We ran into this in https://github.com/python-trio/trio/issues/3053. I'm not certain this is a bug, but it's certainly surprising and I don't remember seeing anything about this while we were adding our custom plugin!
[x] output of
pip list
from the virtual environment you are using[x] pytest and operating system versions
As shown above, pytest is 8.3.2. I'm reproducing this on Windows 11, but it happened in CI for us for every base actions runner OS + Alpine Linux.
pyproject.toml
:test.py
:src/example/__init__.py
:With that in place, run
pip install . pytest
and then these two commands have conflicting results: