pytest has issues when setting env PYTHONBREAKPOINT to "pdbp.set_trace" if capturing output

[hynek]

Hi,

I've been testing pdbp as a replacement for pdbpp whose pytest integration is broken in 3.11, but… it seems like so it pdbp's? At least a breakpoint() gives me an

OSError: pytest: reading from stdin while output is captured! Consider using -s.

``` $ env PYTHONBREAKPOINT=pdbp.set_trace pytest tests/ui/test_ui_e2e.py ================================================================ test session starts ================================================================ platform darwin -- Python 3.11.5, pytest-7.4.2, pluggy-1.3.0 rootdir: /Users/hynek/Work/spot-svc configfile: pyproject.toml plugins: anyio-4.0.0, icdiff-0.8 collected 5 items [31] > /Users/hynek/Work/spot-svc/tests/api/test_api_e2e.py(42) .. 34 def test_docs(aclient): 35 """ 36 The docs are served if the client is authentificated. 37 """ 38 resp = aclient.get("/") 39 40 breakpoint() 41 42 -> assert 200 == resp.status_code (Pdb+) ----------------------------------------------------------------- Captured log call ----------------------------------------------------------------- INFO httpx:_client.py:1013 HTTP Request: GET http://localhost/api/ "HTTP/1.1 200 OK" ------------------------------------------------------------- Captured stdout teardown -------------------------------------------------------------- closing 'sqlalchemy.engine.base.Connection' closed 'sqlalchemy.engine.base.Connection' --------------------------------------------------------------- Captured log teardown --------------------------------------------------------------- DEBUG svcs:_core.py:347 closing 'sqlalchemy.engine.base.Connection' [33] > /Users/hynek/Work/spot-svc/tests/api/test_api_e2e.py(42) .. 34 def test_docs(aclient): 35 """ 36 The docs are served if the client is authentificated. 37 """ 38 resp = aclient.get("/") 39 40 breakpoint() 41 42 -> assert 200 == resp.status_code (Pdb+) ----------------------------------------------------------------- Captured log call ----------------------------------------------------------------- INFO httpx:_client.py:1013 HTTP Request: GET http://localhost/api/ "HTTP/1.1 200 OK" ------------------------------------------------------------- Captured stdout teardown -------------------------------------------------------------- closing 'sqlalchemy.engine.base.Connection' closed 'sqlalchemy.engine.base.Connection' --------------------------------------------------------------- Captured log teardown --------------------------------------------------------------- DEBUG svcs:_core.py:347 closing 'sqlalchemy.engine.base.Connection' DEBUG svcs:_core.py:349 closed 'sqlalchemy.engine.base.Connection' ============================================================== short test summary info ============================================================== FAILED tests/api/test_api_e2e.py::test_docs - OSError: pytest: reading from stdin while output is captured! Consider using `-s`. =========================================================== 1 failed, 47 passed in 10.28s =========================================================== ```

I've tried importing pdbp in my contest.py instead of setting PYTHONBREAKPOINT to the same result.

Has something changed again or am I missing something?

$ pip list | grep pdbp
pdbp                      1.5.0

[mdmintz]

A pytest.ini file with the following should resolve your issue:

[pytest]

# Display console output.
addopts = --capture=no

Or running pytest with -s: pytest -s.

Not sure if you're overriding something somewhere, but your default configuration is capturing console output instead of displaying it. (Your output had: pytest: reading from stdin while output is captured! Consider using "-s") The console output must be displayed for the pdbp output to appear inside a pytest run.

It was probably caused when you prepended env PYTHONBREAKPOINT=pdbp.set_trace to your pytest run command instead of having import pdbp in an __init__.py file to load it. When using that config with pytest, definitely add the -s (or the pytest.ini file).

I mostly use pdbp with https://github.com/seleniumbase/SeleniumBase, which runs with pytest, and works normally.

[hynek]

Yeah, but capturing is the default (and I want it that way; otherwise I get a lot of output from logs :)) and that used to work with pdbpp. That's what https://github.com/pdbpp/pdbpp/issues/519 is about.

You could capture output as by default, and it somehow detected dropping into the debugger. Last time I checked it worked with the current master branch (again) so inspiration could be borrowed.

I'm not gonna tell you what to do, but it would be really cool to have a maintained debugger that supports this. 😇

[mdmintz]

I think https://github.com/pytest-dev/pytest/issues/5498#issuecomment-744038844 is the real issue:

Fortunately, there are multiple workarounds:

• pytest -s
• addopts = --capture=no in a pytest.ini file under [pytest].
• import pdbp in __init__.py instead of env PYTHONBREAKPOINT=pdbp.set_trace when using pytest.

[hynek]

Look, I'm not gonna argue with you, it's you prerogative and your project. But some clarifications for posterity:

• pytest -s / adopts = --capture=no perfectly works with the current pdbpp version on PyPI. It's absolutely not an acceptable solution for my projects, though. This is a central pytest feature many people rely on.
• As I wrote, I have tried import pdbpp. What I arguably didn't write is that it fails the same way at the current pdbpp PyPI version (again, last time I tried the master version, it worked):

IOW it shows up, but doesn't allow me to enter commands. Maybe the fix came after your fork, I don't know.

Anyhow, sorry for the noise!

[mdmintz]

The difference is that the external pdbpp repo completely overrides pdbp just by having it installed.

But with pdbp (this repo), you need to put import pdbp into an __init__.py file for that same behavior. We give people the option to not always override the built-in pdb, but in this case, you should override it. Put import pdbp into an __init__.py file, and you'll get the working functionality without changing the PYTHONBREAKPOINT env variable.

[hynek]

Which I did, as shown in the screenshot above.

Maybe I should be clearer here, because it turned out to be two different problems:

1. if I use PYTHONBREAKPOINT, I get the error from my first post (OSError: pytest: reading from stdin while output is captured! Consider using -s).
2. if I import pdbp (without setting -s), I get the hanging debugger as shown in the screenshot where I have to use ctrl-d to get out of the session.

Point two is the exact same behavior as with PyPI pdbpp when I forget to pass -s and the reason I'm looking for alternatives.

Again, I'm not trying to make you to do anything. I just feel I must be communicating in a confusing way.

[mdmintz]

I see how to fix this for captured output. I'll ship a new release shortly.

[hynek]

To be clear one last time – because I'm not sure about the rename of the issue: I'm 100% OK with pdbp's pytest integration only working after a import pdbp. It's just that currently it doesn't. 😅 It will give you a colorful-but-broken debug session as you can see in the screenshot above.

[mdmintz]

The only issue I see in the one where setting env PYTHONBREAKPOINT to pdbp.set_trace leads to "OSError: pytest: reading from stdin while output is captured! Consider using -s." when not adding -s to display the output.

Otherwise, pdbp works fine with pytest. Here's an example where I use pytest --trace to immediately open a breakpoint when starting a test: (Have import pdbp in an __init__.py file):

Perhaps the change I'm about to make will fix your other issues as well.

[hynek]

Uh this is very odd.

Let's use structlog as an example since it's a public project.

If I apply this:

diff --git src/structlog/__init__.py src/structlog/__init__.py
index 2ffe2c8..c5c3053 100644
--- src/structlog/__init__.py
+++ src/structlog/__init__.py
@@ -6,6 +6,8 @@

 from __future__ import annotations

+import pdbp
+
 from structlog import (
     contextvars,
     dev,

to the current main branch and run pytest --trace I get this:

If I run pytest --trace -s I get this:

Please not the (pdb+) prompt that is missing from the previous screenshot.

I have the same behavior with the current master checkout of pdbp.

I have renames both my ~/.pdbrc and ~/.pdbrc.py files.

Are you sure that project doesn't have an addopts = -s?

[mdmintz]

Looks like no change in pdbp is necessary.

See https://stackoverflow.com/a/61149394/7058266

pytest --capture=tee-sys was recently added (see https://docs.pytest.org/en/stable/changelog.html). Not only can you capture the output of stdout and stderr, but you can also display that output at the same time.

Being able to see the output is necessary for pdbp to function correctly. The -s option already gave you that, but then it stopped the capture, which may be needed by some things (sounds like you need it). Now you can have it both ways. Enable it via the command-line option (or via the pytest.ini file). You'll still want import pdbp in an __init__.py file, as mentioned in the ReadMe.