Read Python 2 coverage in Python 3

[Spectre5]

Have you asked elsewhere?

No.

Describe your situation

I have a situation where I need to run coverage on some older Python 2.7 code (it runs on a legacy closed system and I sadly cannot upgrade it to Python 3 at this time) as well as some Python 3 code. Thus I'd like to run coverage combine and coverage report from a Python 3 environment. For the most part, this seems to work. However, I've found that the def line from Python 2.7 code that is decorated with @property gets reported as not being covered.

I suppose this may be due to some difference in the SQL file produced by 2.7 and then read in 3.x. Perhaps I'm foolish for expecting that this may work with the changes between 2.7 and 3.x. Would you expect this to work? I've boiled it down to a minimal working example provided below and verified the behavior on both Windows and Linux.

The directory layout is:

issue
    |-- src
        |-- dummy.py
        |-- __init__.py  (empty file)
    |--- tests
        |-- test_dummy.py
    noxfile.py
    .coveragerc

where nox has been used to simplify the multiple environments. The contents of each file is:

dummy.py:

class SomeClass:
    def __init__(self):
        self.x = 5.0

    @property
    def prop_x(self):
        return self.x

test_dummy.py

from dummy_pkg.dummy import SomeClass
def test_dummy():
    assert SomeClass().prop_x == 5.0

.coveragerc

[run]
source = src/dummy_pkg

[report]
show_missing = true

noxfile.py

from pathlib import Path

import nox

# virtualenv stopped supporting creation of Python 2.7 virtual environments in
# version 20.22.0, so make sure that the version in the nox pipx environment
# is older than that by running:
#   pipx inject nox 'virtualenv<20.22.0' --force

@nox.session(python='2.7')
def test(session: nox.Session) -> None:
    session.install('--upgrade', 'pip')
    # coverage 5.5 is the last to support Python 2.7
    session.install('pytest', 'coverage==5.5.0')
    # add src directory to the path so dummy_pkg is found
    start = Path(session.virtualenv.location).parent.parent
    env = {'PYTHONPATH': str(start / 'src')}
    session.run('coverage', 'run', '-m', 'pytest', env=env)
    session.notify('coverage')

# other Python 3 tests might happen here...

@nox.session(python='3.10')
def coverage(session: nox.Session) -> None:
    session.install('--upgrade', 'pip')
    session.install('coverage==5.5.0')  # same version as used in Python 2.7
    session.run('coverage', 'report')

Run this from the top level using the follow (rm used to clean the directory on each run):

rm -rf .nox .coverage .pytest_cache && nox

The result coverage reported from this is the following, where line 6 is def prop_x(self):

nox > coverage report
Name                        Stmts   Miss  Cover   Missing
---------------------------------------------------------
src/dummy_pkg/__init__.py       0      0   100%
src/dummy_pkg/dummy.py          7      2    71%   6
---------------------------------------------------------
TOTAL                           7      2    71%

However, if I instead use python='2.7' for the coverage session in nox, then the coverage is correctly reported as 100%:

nox > coverage report
Name                        Stmts   Miss  Cover   Missing
---------------------------------------------------------
src/dummy_pkg/__init__.py       0      0   100%
src/dummy_pkg/dummy.py          5      0   100%
---------------------------------------------------------
TOTAL                           5      0   100%

[nedbat]

Python has changed between versions about how it reports lines being executed. I suspect this is the reason for the def line appearing and disappearing. If you have flexibility about what version of Python to use, use the one that gives you the results you like.

Sorry if I've missed something here.

[Spectre5]

You probably haven't missed anything and something along these lines is what I was expected. I was hoping there might be some work around to it that you were aware of. I didn't include it in this MWE, but in my actual code I also run coverage on some Python 3 code, so I was hoping to use Python 3 for the coverage session where I'd also combine the coverage before reporting it.

[nedbat]

Have you tried combining the two runs together and then reporting on the combined data?

[Spectre5]

If I test the same code in Python 2 and 3 and then combine it, then I get 100% coverage, as expected. However, in my larger application, some of the code is only run on Python 2.7, so when I combine and report with that I get those missing lines as indicated above.

However, it does seem to work if I just use Python 2.7 for the coverage step (to combine and report). So I'll just do that, though in this project there is very little Python 3 code so perhaps this solution would also have issues with some aspects of Python 3. But at least it works for me at the moment.