Open mariuswallraff opened 7 months ago
So you're saying using filterwarnings makes your coverage drop to 0%? Can't really help without a reproducer.
Here’s a reproducer: https://github.com/flying-sheep/pytest-cov-filterwarnings
As its readme says, the cause is that pytest
imports warning classes referenced in filterwarnings
before pytest-cov is initialized.
FWIW, I believe this is connected to https://github.com/pytest-dev/pytest-cov/issues/437
Ah yeah, seems like this issue is a duplicate of #437 then.
As @ionelmc says in https://github.com/pytest-dev/pytest-cov/issues/635#issuecomment-2012514538: This is a chicken-and-egg problem. Also pytest documents its plugin order: it loads builtin plugins first, e.g. the warnings
plugin, which has a hook that runs immediately: pytest_load_initial_conftests
.
So I guess the only way to prevent that is to pass -p no:warnings
.
Now the question is, if we do that, can we re-enable it at runtime via pytest_plugins = ["warnings"]
?
Summary
First of all sorry for this incomplete bug report (no reproducer). Feel free to close it, but I figured it might still help somebody.
The problem is that the coverage data collection fails if the
filterwarnings
option inpytest.ini
is used to ignore a custom warning defined in our project, but only under Linux (Debian and Alpine) and Mac; it works fine under Windows with the nominally same conda environment and plugin versions, and it works fine as soon as the option is commented out.I sadly cannot tell you if/which additional conditions play into this.
Expected vs actual result
expected:
TOTAL 17353 5185 70%
actual:
TOTAL 17353 17353 0%
with this showing at the end of running the tests (or once for every worker if usingpytest-xdist
):Reproducer
Our pytest call:
where
XXX
is a subfolder of the working directory, and the top level of our package.Our
pytest.ini
:(some arguments are doubled/overridden because one is running in GitLab's CI/CD pipeline, while the
ini
is also used for local execution, but I played around with those options and they are not at fault)Our
.coveragerc
:Not a real reproducer, sorry, but I tried various different combinations of options (including disabling
xdist
), and the only thing that helped was commenting outfilterwarnings
.Versions
These versions are identical on the different systems (i.e., between the unix ones where this fails with
filterwarnings
active, and Windows, where it works).The problem is very likely also true for versions that are about a year old, because that was when we first encountered the problem (but thought it was an incompatibility with
xdist
).Config
tox.ini
is not used,setup.cfg
does not contain testing-related information, for the rest see aboveCode
Sorry, can't share our code, and didn't create a minimal example.