When tests write to stdout/stderr the text ends up writing immediately to the terminal and pytest is unable to intercept it (with the capsys fixture) for inspection. Ideally pytest-codspeed wouldn't intercept stdout/stderr. I have tested this with both Python 3.12 and Python 3.11, both result in the same issue.
To reproduce:
from __future__ import annotations
import sys
import pytest
from pytest import CaptureFixture
@pytest.mark.benchmark
def test_print():
"""Test print statements are captured by pytest (i.e., not printed to terminal in
the middle of the progress bar) and only displayed after test run (on failures)."""
print("print to stdout")
print("print to stderr", file=sys.stderr)
@pytest.mark.benchmark
def test_capsys(capsys: CaptureFixture):
"""Test print statements are captured by capsys (i.e., not printed to terminal in
the middle of the progress bar) and can be inspected within test."""
print("print to stdout")
print("print to stderr", file=sys.stderr)
stdout, stderr = capsys.readouterr()
assert stdout == "print to stdout\n"
assert stderr == "print to stderr\n"
When tests write to
stdout
/stderr
the text ends up writing immediately to the terminal and pytest is unable to intercept it (with thecapsys
fixture) for inspection. Ideallypytest-codspeed
wouldn't interceptstdout
/stderr
. I have tested this with both Python 3.12 and Python 3.11, both result in the same issue.To reproduce:
See https://github.com/kenodegard/pytest-codspeed/pull/1 for benchmark run, screenshot below for posterity: