So I noticed the existence of https://pytest-cov.readthedocs.io/en/latest/markers-fixtures.html, and it occurred to me that it'd be useful to have a fixture or a mark that would make Coverage.py exclude the entire test instead of having to append # pragma: no cover.
The use case is failing tests contributed, marked as xfail and merged: https://blog.ganssle.io/articles/2021/11/pytest-xfail.html. Oftentimes, these are failing early until fixed, which interrupts the control flow of such test functions, meaning that the rest (probably containing a number of asserts) are never going to report any coverage (until the thing is fixed and xfail dropped, usually).
I'd like to encourage people to submit such xfailing tests with bug reports without imposing a penalty on the coverage metric.
So I noticed the existence of https://pytest-cov.readthedocs.io/en/latest/markers-fixtures.html, and it occurred to me that it'd be useful to have a fixture or a mark that would make Coverage.py exclude the entire test instead of having to append
# pragma: no cover.The use case is failing tests contributed, marked as
xfailand merged: https://blog.ganssle.io/articles/2021/11/pytest-xfail.html. Oftentimes, these are failing early until fixed, which interrupts the control flow of such test functions, meaning that the rest (probably containing a number of asserts) are never going to report any coverage (until the thing is fixed and xfail dropped, usually).I'd like to encourage people to submit such xfailing tests with bug reports without imposing a penalty on the coverage metric.
I'm not sure if this is currently possible to implement in pytest-cov, though. The only Coverage.py API I've found that allows marking lines for exclusion is the plugins one: https://coverage.readthedocs.io/en/7.3.2/api_plugin.html#coverage.FileReporter.excluded_lines.