pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
12.07k stars 2.67k forks source link

Output of subprocess not captured when fixture started by doctests #3752

Open jaraco opened 6 years ago

jaraco commented 6 years ago

When invoking a fixture with doctests, output isn't captured, so output from subprocesses can be seen in the output even in subsequent tests. Consider this test:

import sys
import subprocess

import pytest

@pytest.fixture(scope='session')
def emits_stdout():
    subprocess.run([
        sys.executable,
        '-c', ';'.join((
            'import time',
            'print("hello world")',
            'time.sleep(0.5)',
            'print("hello whirled")',
        ))
    ])

def func():
    """
    >>> getfixture('emits_stdout')
    """

def test_something(emits_stdout):
    import time
    time.sleep(0.7)
    assert True

Invoked emits:

draft $ rwt pytest -- -m pytest --doctest-modules test-issuenn.py
Collecting pytest
  Using cached https://files.pythonhosted.org/packages/9e/a1/8166a56ce9d89fdd9efcae5601e71758029d90e5644e0b7b6eda07e67c35/pytest-3.7.0-py2.py3-none-any.whl
Collecting py>=1.5.0 (from pytest)
  Using cached https://files.pythonhosted.org/packages/f3/bd/83369ff2dee18f22f27d16b78dd651e8939825af5f8b0b83c38729069962/py-1.5.4-py2.py3-none-any.whl
Collecting more-itertools>=4.0.0 (from pytest)
  Using cached https://files.pythonhosted.org/packages/79/b1/eace304ef66bd7d3d8b2f78cc374b73ca03bc53664d78151e9df3b3996cc/more_itertools-4.3.0-py3-none-any.whl
Collecting attrs>=17.4.0 (from pytest)
  Using cached https://files.pythonhosted.org/packages/41/59/cedf87e91ed541be7957c501a92102f9cc6363c623a7666d69d51c78ac5b/attrs-18.1.0-py2.py3-none-any.whl
Collecting atomicwrites>=1.0 (from pytest)
  Using cached https://files.pythonhosted.org/packages/0a/e8/cd6375e7a59664eeea9e1c77a766eeac0fc3083bb958c2b41ec46b95f29c/atomicwrites-1.1.5-py2.py3-none-any.whl
Collecting six>=1.10.0 (from pytest)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting pluggy>=0.7 (from pytest)
  Using cached https://files.pythonhosted.org/packages/f5/f1/5a93c118663896d83f7bcbfb7f657ce1d0c0d617e6b4a443a53abcc658ca/pluggy-0.7.1-py2.py3-none-any.whl
Collecting setuptools (from pytest)
  Using cached https://files.pythonhosted.org/packages/ff/f4/385715ccc461885f3cedf57a41ae3c12b5fec3f35cce4c8706b1a112a133/setuptools-40.0.0-py2.py3-none-any.whl
Installing collected packages: py, six, more-itertools, attrs, atomicwrites, pluggy, setuptools, pytest
Successfully installed atomicwrites-1.1.5 attrs-18.1.0 more-itertools-4.3.0 pluggy-0.7.1 py-1.5.4 pytest-3.7.0 setuptools-40.0.0 six-1.11.0
===================================================================== test session starts =====================================================================
platform darwin -- Python 3.7.0, pytest-3.7.0, py-1.5.4, pluggy-0.7.1
rootdir: /Users/jaraco/draft, inifile:
plugins: xonsh-0.6.7
collected 2 items

test-issuenn.py hello world
hello whirled
..                                                                                                                                      [100%]

================================================================== 2 passed in 1.28 seconds ===================================================================

If you comment out the doctest, the output does not appear in the output.

RonnyPfannschmidt commented 6 years ago

my understanding is that doctest can't capture without breaking itself - its not clear what to do about sub-processes there

jaraco commented 6 years ago

Is it possible to, in getfixture, as exposed in doctests, always restore output capturing, invoke the fixture, then disable output capturing again?