Line profiler with pytest-cov

The LineProfiler.enable_by_count() function disrupts the behavior of the pytest-cov plugin. Lines that are hit after the profiler is enabled are not reported as covered by the plugin.

As a simple working example, suppose I have a file run_profiler.py which provides a function set_profiler, and a test script that runs it:

run_profiler.py

from line_profiler import LineProfiler

prof = None

def set_profiler(enable=False):
    global prof
    prof = LineProfiler()

    if enable:
        prof.enable_by_count()
        print("Profiler enabled")
    else:
        print("Profiler not enabled")

test_prof.py

from prof_runner import set_profiler
import pytest

def test_line_profiler():
    set_profiler(False)
    set_profiler(True)

I run the test file with the following line:

pytest --cov=prof_runner --cov-report html:./coverage -s

This runs the test file and generates a coverage report in html. Trivially, this test should cover all of the lines. However, I'm finding that any lines that are run after prof.enable_by_count() are missed. These are screenshots of the coverage reports, with red highlighting the lines that are not covered. In one case (coverage_with_falsefirst.png), the test is run as written above. In the other case, set_profiler is first run with enable=True and then with enable=False.

With enable=False first: coverage_with_falsefirst

With enable=True first: coverage_with_truefirst

Note that in the case that enable_by_count is run on the first call to set_profiler, the lines in the enable=False block are missed.

However, in both calls, the output from pytest looks like this: output

So both lines are definitely being run, but somehow line_profiler is disrupting the coverage plugin.

rkern / line_profiler

Line profiler with pytest-cov #179