nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
3.01k stars 433 forks source link

Incorrect branch coverage in Python 3.9 #1177

Closed chrisrossi closed 3 years ago

chrisrossi commented 3 years ago

There is a very specific case that exposes a bug in branch coverage only for Python 3.9. Here is a testcase I've distilled down to be as simple as I can get it and still expose the error:

import contextlib

@contextlib.contextmanager
def context():
    yield None

def coroutine():
    with context():
        try:
            try:
                yield "yes"

            except Exception as e:
                raise e

        finally:
            pass

        return("hello")

task = coroutine()
assert list(task) == ["yes"]

error = Exception()
task = coroutine()
assert "yes" == next(task)

thrown = False
try:
    task.throw(Exception, error, error.__traceback__)
except Exception:
    thrown = True

assert thrown

All elements seem to be required here: the coroutine, the context manager, the try/except nested inside a try/finally, and the return statement from inside the context manager.

Running coverage in Python 3.9:

(coverage-bug) chris@spirit:~/proj/google/ndb$ coverage run coverage_bug.py && coverage report
Name              Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------------
coverage_bug.py      24      0      2      1    96%   19->exit
-------------------------------------------------------------
TOTAL                24      0      2      1    96%
Coverage failure: total of 96 is less than fail-under=100

Running coverage in Python 3.8:

(coverage-bug-3.8) chris@spirit:~/proj/google/ndb$ coverage run coverage_bug.py && coverage report
Name              Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------------
coverage_bug.py      24      0      2      0   100%
-------------------------------------------------------------
TOTAL                24      0      2      0   100%

Note that deindenting the return statement so that it is outside of the context manager causes coverage to reach 100% again, which in the actual use case this came up in is an acceptable work-around.

nedbat commented 3 years ago

This looks like the same issue as #1175 (a final "pass" is traced incorrectly). Since 3.10 handles it correctly, I don't think it will be fixed in the older versions of Python, but I've written https://bugs.python.org/issue44672 to get a decision.