kvas-it / pytest-console-scripts

Pytest plugin for testing console scripts
MIT License
78 stars 14 forks source link

`pytest-console-scripts` causes `pytest-xdist` tests to fail #85

Open mtelka opened 12 months ago

mtelka commented 12 months ago

I tried to run tests for pytest-xdist 3.4.0 with pytest-console-scripts 1.4.1 installed and disabled using -p no:console-scripts set in the PYTEST_ADDOPTS environment variable. And, surprisingly, pytest-xdist tests fails:

============================= test session starts ==============================
platform sunos5 -- Python 3.9.16, pytest-7.4.3, pluggy-1.3.0 -- $(BUILD_DIR)/.tox/py39/bin/python
cachedir: .tox/py39/.pytest_cache
rootdir: $(BUILD_DIR)
configfile: tox.ini
testpaths: testing
plugins: xdist-3.4.0
collecting ... collected 203 items

testing/acceptance_test.py::TestDistribution::test_n1_pass PASSED        [  0%]
testing/acceptance_test.py::TestDistribution::test_n1_fail FAILED        [  0%]

=================================== FAILURES ===================================
________________________ TestDistribution.test_n1_fail _________________________

self = <acceptance_test.TestDistribution object at 0x7fffadcb2760>
pytester = <Pytester PosixPath('/tmp/pytest-of-marcel/pytest-79/test_n1_fail0')>

    def test_n1_fail(self, pytester: pytest.Pytester) -> None:
        p1 = pytester.makepyfile(
            """
            def test_fail():
                assert 0
        """
        )
        result = pytester.runpytest(p1, "-n1")
>       assert result.ret == 1
E       assert <ExitCode.USAGE_ERROR: 4> == 1
E        +  where <ExitCode.USAGE_ERROR: 4> = <RunResult ret=ExitCode.USAGE_ERROR len(stdout.lines)=0 len(stderr.lines)=5 duration=0.03s>.ret

$(BUILD_DIR)/testing/acceptance_test.py:32: AssertionError
----------------------------- Captured stderr call -----------------------------
ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...]
__main__.py: error: unrecognized arguments: -n1
  inifile: None
  rootdir: /tmp/pytest-of-marcel/pytest-79/test_n1_fail0

=========================== short test summary info ============================
FAILED testing/acceptance_test.py::TestDistribution::test_n1_fail - assert <E...
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!

With pytest-console-scripts uninstalled all pytest-xdist tests pass.

HexDecimal commented 12 months ago

Are you sure -n1 is correct? The pytest-xdist docs seem to show the syntax as: -n 1

mtelka commented 12 months ago

I think it is okay. The test_n1_pass test have -n1 too and it pass. Also, when I run without pytest-console-scripts installed all pytest-xdist tests pass.

mtelka commented 12 months ago

Hmmm, maybe the problem is in pytest-xdist itself because I found that it fails similarly with pytest-checkdocs or pytest-randomly plugins installed (and disabled). Interestingly, pytest-xdist does not fail with pytest-black or pytest-mypy installed.