pytest-dev / pytest-forked

extracted --boxed from pytest-xdist to ensure backward compat
MIT License
65 stars 21 forks source link

Silent failures when a worker dies (e.g. segfault) and using forked and xdist #27

Open maksimt opened 5 years ago

maksimt commented 5 years ago

The following test case fails when both --forked and xdist (-n, 2) is used. It passes if only --forked is used.

import pytest

def test_segfault_reported_as_failure(testdir):
    p1 = testdir.makepyfile("""
    import ctypes

    def test_with_segfault():
        ctypes.string_at(0)

    def test_that_passes():
        assert True
    """)
    result = testdir.runpytest(p1, '--forked', '-n', 2)  # this doesn't fail if we remove '-n', 2
    result.stdout.fnmatch_lines([
        "*1 failed*"
    ])

with the following output:

======================================== test session starts =========================================
platform linux -- Python 3.7.5, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /home
plugins: xdist-1.30.1.dev1+g79dd52b, forked-1.1.3
collected 1 item

home/test_segfault.py Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/_pytest/pytester.py", line 863, in runpytest_inprocess
    reprec = self.inline_run(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/_pytest/pytester.py", line 829, in inline_run
    ret = pytest.main(list(args), plugins=plugins)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 71, in main
    config = _prepareconfig(args, plugins)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 221, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/local/lib/python3.7/site-packages/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 736, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 943, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 878, in _preparse
    self._initini(args)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/__init__.py", line 803, in _initini
    args, namespace=copy.copy(self.option)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 128, in parse_known_and_unknown_args
    return optparser.parse_known_args(args, namespace=namespace)
  File "/usr/local/lib/python3.7/argparse.py", line 1787, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/local/lib/python3.7/argparse.py", line 1828, in _parse_known_args
    option_tuple = self._parse_optional(arg_string)
  File "/usr/local/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 377, in _parse_optional
    if not arg_string[0] in self.prefix_chars:
TypeError: 'int' object is not subscriptable
F

My execution environment is in the following docker container:

FROM python:3.7-alpine

RUN apk add --no-cache --virtual .build-deps git

RUN pip3 install git+https://github.com/pytest-dev/pytest-forked

RUN pip3 install git+https://github.com/pytest-dev/pytest-xdist
csymeonides-mf commented 4 years ago

Not sure if this is the same issue, but when using pytest with --boxed, test failures in subTests are not reported.