"ValueError: I/O operation on closed file." with click's CliRunner (capturing)

[blueyed]

With click's CliRunner I've noticed that it throws a ValueError when pytest's pdb gets used in the CliRunner.isolation context:

test_pytest_capture.py:

def test_click_isolation():
    from click.testing import CliRunner

    runner = CliRunner()
    with runner.isolation() as out:
        print('isolation')
        __import__('pdb').set_trace()
        print('END')
        print(out, out.getvalue())

Test run:

% pytest test_pytest_capture.py
==================================== test session starts ====================================
platform linux -- Python 3.6.4, pytest-3.5.1.dev7+ged118d7f, py-1.5.3, pluggy-0.6.0
rootdir: …/Vcs/click, inifile:
collected 1 item                                                                            

test_pytest_capture.py 
>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>
> …/Vcs/click/test_pytest_capture.py(8)test_click_isolation()
-> print('END')
(Pdb) c
END
F                                                              [100%]

========================================= FAILURES ==========================================
___________________________________ test_click_isolation ____________________________________

    def test_click_isolation():
        from click.testing import CliRunner

        runner = CliRunner()
        with runner.isolation() as out:
            print('isolation')
            __import__('pdb').set_trace()
>           print('END')
E           ValueError: I/O operation on closed file.

test_pytest_capture.py:8: ValueError
================================= 1 failed in 0.77 seconds ==================================

When using -s bdb.BdbQuit is raised:

% pytest test_pytest_capture.py -s
==================================== test session starts ====================================
platform linux -- Python 3.6.4, pytest-3.5.1.dev7+ged118d7f, py-1.5.3, pluggy-0.6.0
rootdir: …/Vcs/click, inifile:
collected 1 item                                                                            

test_pytest_capture.py F

========================================= FAILURES ==========================================
___________________________________ test_click_isolation ____________________________________

    def test_click_isolation():
        from click.testing import CliRunner

        runner = CliRunner()
        with runner.isolation() as out:
            print('isolation')
            __import__('pdb').set_trace()
>           print('END')

test_pytest_capture.py:8: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_pytest_capture.py:8: in test_click_isolation
    print('END')
/usr/lib/python3.6/bdb.py:48: in trace_dispatch
    return self.dispatch_line(frame)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pdb.Pdb object at 0x7f35b42e8710>, frame = <frame object at 0x7f35b429f848>

    def dispatch_line(self, frame):
        if self.stop_here(frame) or self.break_here(frame):
            self.user_line(frame)
>           if self.quitting: raise BdbQuit
E           bdb.BdbQuit

/usr/lib/python3.6/bdb.py:67: BdbQuit
================================= 1 failed in 0.06 seconds ==================================

The ValueError comes from out.getvalue really, the location is off due to the pdb.set_trace (see https://github.com/pytest-dev/pytest/issues/3237).

pytest.testing.isolation: https://github.com/pallets/click/blob/55682f6f5348f5220a557f89c3a796321a52aebf/click/testing.py#L139

[pytestbot]

GitMate.io thinks possibly related issues are https://github.com/pytest-dev/pytest/issues/14 (ValueError: I/O operation on closed file), https://github.com/pytest-dev/pytest/issues/2370 (IOError: close() called during concurrent operation on the same file object.), https://github.com/pytest-dev/pytest/issues/1873 (with capsys.disabled() causes "I/O operation on closed file"), https://github.com/pytest-dev/pytest/issues/3174 (pytest 3.4 breaks test with CherryPy - no attribute 'testscollected' & I/O operation on closed file), and https://github.com/pytest-dev/pytest/issues/1585 (Capture is all wrong).

[blueyed]

I've thought that https://github.com/pytest-dev/pytest/pull/2619 might have helped here, but it does not - but without having investigated much it might go in the right direction.

It looks a bit like pytest should maybe restore / handle the changed output stream(s) from the outer context manager, but uses the one(s) from before the test started.

[blueyed]

Ref: https://github.com/pallets/click/issues/654

[jwillis0720]

@blueyed why did you close this?

[itobysq]

@blueyed how did you resolve this issue? I am facing the same problem.

[raphaelboudreault]

@blueyed Was this resolved? Why is this closed?

[elasticdotventures]

@blueyed Was this resolved? Why is this closed?

This nccr-itmo/FEDOT#765 was solved this by disabling the logging in stdout. the issue is related to how logging is handled between click & pytest

a potential resolution to pytest cli is to add the "--capture=no" or "-s" (note: -s is a shortcut) parameter which will disable log capturing.

[Solvero]

I followed suggestion to remove CliRunner. Seems that it works this way (link to docs):

with function_under_test.make_context('function_under_test', ['arg1', 'arg2']) as ctx:
  result = function_under_test.invoke(ctx)

[snirbenyosef]

@Solvero result is allways None for you to ?

does someone found a way to fix the issue? or to overcome it?

[Solvero]

@snirbenyosef I've just checked. I have correct value returned (in my case its integer).

Please double check that you return value in 'function_under_test'.

[snirbenyosef]

@Solvero I meant I want the stdout and the exit_code, is it possible?

[Solvero]

For that I would use subprocess instead:

resp = subprocess.check_output(...)
assert resp.returncode == 0
assert resp.stdout ....