adamjstewart commented 2 years ago

Summary

I believe I've discovered a heisenbug. I have code with a structure like:

for i in x:
    if i == 'foo':
        continue

When I run pytest --cov on this file, I'm told that the continue line does not have coverage. However, when I try to investigate this:

for i in x:
    if i == 'foo':
        print('skipped')
        continue

all of a sudden I have 100% coverage.

Expected vs actual result

I wouldn't expect the presence or absence of a print statement to change whether or not the continue line is covered by my tests.

Reproducer

I'm struggling to find a good minimal reproducible example of this. For now, see the links to the PR where this was first discovered below.

Versions

Python 3.7–3.9
pytest 6.2.5, 7.1.1
pytest-cov 3.0.0
pluggy 1.0.0
nbmake 0.5, 1.3.0
OS: Linux, macOS, Windows

Config

pyproject.toml:

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "@overload",
]

Code

This was initially discovered when working on the following PR: https://github.com/microsoft/torchgeo/pull/507. For commit https://github.com/microsoft/torchgeo/pull/507/commits/1dd06b5597d2276b2dc715ffe4b65eafbde4de8d, the tests fail due to missing coverage of the continue statement. When I add a print statement (https://github.com/microsoft/torchgeo/pull/507/commits/b12efc1fb6ea36fff85ea05c9ea44f21a36baa33), all of a sudden the problem disappears and all lines are covered. I'm also able to reproduce this locally.

hoefling commented 2 years ago

Probably same issue as in https://github.com/pytest-dev/pytest-cov/issues/368?