Pytest ipdb starts shell one frame too deep if coverage >= 6.4.1

[khink]

Not sure if this is a bug in coverage itself, but as pinning coverage to 6.4 fixed it for us, i'll ask here first.

When i run pytest with coverage 6.4.1 or 6.4.2, putting a breakpoint in the code (or in the test code) will start a shell one level deeper than where the breakpoint is. To return to the breakpoint location, i have to type up.

Test code:

import pytest

pytestmark = pytest.mark.django_db

def test_view_homepage(django_app):
    response = django_app.get("/")
    breakpoint()
    assert response.status_code == 200

(With the shell env PYTHONBREAKPOINT=ipdb.set_trace)

Running the test with the breakpoint:

$ pytest -k test_view_homepage
Test session starts (platform: linux, Python 3.9.13, pytest 7.1.2, pytest-sugar 0.9.5)
cachedir: .pytest_cache
django: settings: my_project.settings.test (from ini)
rootdir: /home/kees/Projects/myproject, configfile: setup.cfg, testpaths: my_project, site_settings
plugins: mock-3.8.2, freezegun-0.4.2, sugar-0.9.5, django-4.5.2, black-0.3.12, Faker-13.15.0, flake8-1.1.1, isort-3.0.0, factoryboy-2.5.0, django-constance-2.9.0, xdist-2.5.0, forked-1.4.0, django-webtest-1.9.10, cov-3.0.0
collecting ... Creating test database for alias 'default'...
--Call--
> /home/kees/Projects/myproject/env/lib/python3.9/site-packages/django_webtest/response.py(17)status_code()
     16 
---> 17     @property
     18     def status_code(self):

ipdb> up
> /home/kees/Projects/myproject/my_project/tests/test_view_homepage.py(9)test_view_homepage()
      8     breakpoint()
----> 9     assert response.status_code == 200
     10 

ipdb> 

I'd expect to get a shell at the breakpoint location, but what happens is i have to go up one frame to get there. Pinning coverage to 6.4 fixes this. Other used packages are in the attached requirements.txt file.

This may be related to #1402 which also happens in the same version of coverage, and is also about breakpoints.

requirements.txt

[maerteijn]

The same behaviour occurs with the build in pdb, so it is not specifically related to ipdb. When coverage reports with pytest-coverage are disabled, the issue doesn''t occur.

[nedbat]

I tried reproducing the problem with simpler code that didn't involve Django, but a similar structure:

test_code.py:

class Response:
    @property
    def status(self):
        return 17

def test_it():
    response = Response()
    breakpoint()
    assert response.status == 17

But it stops at the same place with pdb or pdb under coverage:

% pytest
================================= test session starts =================================
platform darwin -- Python 3.9.15, pytest-7.2.0, pluggy-1.0.0
rootdir: /System/Volumes/Data/root/src/bugs/bug1420
collected 1 item

test_code.py
>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>
> /System/Volumes/Data/root/src/bugs/bug1420/test_code.py(9)test_it()
-> assert response.status == 17
(Pdb) q

!!!!!!!!!!!!!!!!!!!!!! _pytest.outcomes.Exit: Quitting debugger !!!!!!!!!!!!!!!!!!!!!!!
================================ no tests ran in 2.58s ================================

% coverage run -m pytest
================================= test session starts =================================
platform darwin -- Python 3.9.15, pytest-7.2.0, pluggy-1.0.0
rootdir: /System/Volumes/Data/root/src/bugs/bug1420
collected 1 item

test_code.py
>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>
> /System/Volumes/Data/root/src/bugs/bug1420/test_code.py(9)test_it()
-> assert response.status == 17
(Pdb) q

This is with coverage.py==6.5.0, but I had the same results with 6.4.2. Is there something particular about the Django-involved tests, or did I miss some other important factor here?

[maerteijn]

@nedbat It might be that the pytest-coverage plugin is causing something odd here. I've made a repository with a minimal reproduction use case: https://github.com/maerteijn/coveragepy-1420

[maerteijn]

@nedbat Just ran into this issue again in projects with much newer versions of both coverage and pytest-cov.

Pinning the coverage back to version 6.4.0 solves the issue again. The reproducable repo above still stands.

[nedbat]

Thanks, the reproduction works for me. Looking at the differences between 6.4 and 6.4.1, this line causes the problem:

frame->f_trace_lines = 0;

[maerteijn]

Ah, nice that you could indicate what’s causing this 👍. I think you added this line for a performance increase. Do you think you can solve the issue?

[nedbat]

I guess setting that to zero is telling pdb that the frame shouldn't be involved in debugging also. I wonder if there's a way to have our cake and eat it too?

[khink]

Hi @nedbat,

I wonder if there's a way to have our cake and eat it too?

If that involves a change in pytest-coverage, let me know.

the commit message mentions a "~3% performance increase". I'd just like to add that i prefer that performance penalty over the current state of affairs (which is pinning coverage to 6.4).