search

shawnbrown / datatest

Tools for test driven data-wrangling and data validation.
Other
294 stars 13 forks source link

Crashes pytest-xdist processes (NOTE: See comments for fix.) #51

Closed VasilyevAA closed 3 years ago

VasilyevAA commented 3 years ago

Hi, all! I've got some problem, when start my tests with pytest-xdist

MacOS(Also check in debian) python 3.8.2

pytest==5.4.3 pytest-xdist==1.33.0 datatest==0.9.6

from datatest import accepted, Extra, validate as __validate

def test_should_passed():
    with accepted(Extra):
        __validate({"qwe": 1}, {"qwe": 1}, "")

def test_should_failed():
    with accepted(Extra):
        __validate({"qwe": 1}, {"qwe": 2}, "")

if __name__ == '__main__':
    import sys, pytest
    sys.exit(pytest.main(['/Users/qa/PycharmProjects/qa/test123.py', '-vvv', '-n', '1', '-s']))

Output:

test123.py::test_should_passed 
[gw0] PASSED test123.py::test_should_passed 
test123.py::test_should_failed !!!!!!!!!!!!!!!!!!!! <ExceptionInfo RuntimeError('\'----------------------------------------------------------------------------------------------------\'.../issues\'\n\'----------------------------------------------------------------------------------------------------\'\n') tblen=14>

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/xdist/workermanage.py", line 334, in process_from_remote
INTERNALERROR>     rep = self.config.hook.pytest_report_from_serializable(
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/reports.py", line 355, in pytest_report_from_serializable
INTERNALERROR>     return TestReport._from_json(data)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/reports.py", line 193, in _from_json
INTERNALERROR>     kwargs = _report_kwargs_from_json(reportdict)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/reports.py", line 485, in _report_kwargs_from_json
INTERNALERROR>     reprtraceback = deserialize_repr_traceback(
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/reports.py", line 468, in deserialize_repr_traceback
INTERNALERROR>     repr_traceback_dict["reprentries"] = [
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/reports.py", line 469, in <listcomp>
INTERNALERROR>     deserialize_repr_entry(x) for x in repr_traceback_dict["reprentries"]
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/reports.py", line 464, in deserialize_repr_entry
INTERNALERROR>     _report_unserialization_failure(entry_type, TestReport, reportdict)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/reports.py", line 206, in _report_unserialization_failure
INTERNALERROR>     raise RuntimeError(stream.getvalue())
INTERNALERROR> RuntimeError: '----------------------------------------------------------------------------------------------------'
INTERNALERROR> 'INTERNALERROR: Unknown entry type returned: DatatestReprEntry'
INTERNALERROR> "report_name: <class '_pytest.reports.TestReport'>"
INTERNALERROR> {'$report_type': 'TestReport',
INTERNALERROR>  'duration': 0.002020120620727539,
INTERNALERROR>  'item_index': 1,
INTERNALERROR>  'keywords': {'qa': 1, 'test123.py': 1, 'test_should_failed': 1},
INTERNALERROR>  'location': ('test123.py', 8, 'test_should_failed'),
INTERNALERROR>  'longrepr': {'chain': [({'extraline': None,
INTERNALERROR>                           'reprentries': [{'data': {'lines': ['    def '
INTERNALERROR>                                                               'test_should_failed():',
INTERNALERROR>                                                               '        with '
INTERNALERROR>                                                               'accepted(Extra):',
INTERNALERROR>                                                               '>           '
INTERNALERROR>                                                               '__validate({"qwe": '
INTERNALERROR>                                                               '1}, {"qwe": 2}, '
INTERNALERROR>                                                               '"")',
INTERNALERROR>                                                               'E           '
INTERNALERROR>                                                               'datatest.ValidationError: '
INTERNALERROR>                                                               'does not '
INTERNALERROR>                                                               'satisfy 2 (1 '
INTERNALERROR>                                                               'difference): {',
INTERNALERROR>                                                               'E               '
INTERNALERROR>                                                               "'qwe': "
INTERNALERROR>                                                               'Deviation(-1, '
INTERNALERROR>                                                               '2),',
INTERNALERROR>                                                               'E           }'],
INTERNALERROR>                                                     'reprfileloc': {'lineno': 11,
INTERNALERROR>                                                                     'message': 'ValidationError',
INTERNALERROR>                                                                     'path': 'test123.py'},
INTERNALERROR>                                                     'reprfuncargs': {'args': []},
INTERNALERROR>                                                     'reprlocals': None,
INTERNALERROR>                                                     'style': 'long'},
INTERNALERROR>                                            'type': 'DatatestReprEntry'}],
INTERNALERROR>                           'style': 'long'},
INTERNALERROR>                          {'lineno': 11,
INTERNALERROR>                           'message': 'datatest.ValidationError: does not '
INTERNALERROR>                                      'satisfy 2 (1 difference): {\n'
INTERNALERROR>                                      "    'qwe': Deviation(-1, 2),\n"
INTERNALERROR>                                      '}',
INTERNALERROR>                           'path': '/Users/qa/PycharmProjects/qa/test123.py'},
INTERNALERROR>                          None)],
INTERNALERROR>               'reprcrash': {'lineno': 11,
INTERNALERROR>                             'message': 'datatest.ValidationError: does not '
INTERNALERROR>                                        'satisfy 2 (1 difference): {\n'
INTERNALERROR>                                        "    'qwe': Deviation(-1, 2),\n"
INTERNALERROR>                                        '}',
INTERNALERROR>                             'path': '/Users/qa/PycharmProjects/qa/test123.py'},
INTERNALERROR>               'reprtraceback': {'extraline': None,
INTERNALERROR>                                 'reprentries': [{'data': {'lines': ['    def '
INTERNALERROR>                                                                     'test_should_failed():',
INTERNALERROR>                                                                     '        '
INTERNALERROR>                                                                     'with '
INTERNALERROR>                                                                     'accepted(Extra):',
INTERNALERROR>                                                                     '>           '
INTERNALERROR>                                                                     '__validate({"qwe": '
INTERNALERROR>                                                                     '1}, '
INTERNALERROR>                                                                     '{"qwe": '
INTERNALERROR>                                                                     '2}, "")',
INTERNALERROR>                                                                     'E           '
INTERNALERROR>                                                                     'datatest.ValidationError: '
INTERNALERROR>                                                                     'does not '
INTERNALERROR>                                                                     'satisfy 2 '
INTERNALERROR>                                                                     '(1 '
INTERNALERROR>                                                                     'difference): '
INTERNALERROR>                                                                     '{',
INTERNALERROR>                                                                     'E               '
INTERNALERROR>                                                                     "'qwe': "
INTERNALERROR>                                                                     'Deviation(-1, '
INTERNALERROR>                                                                     '2),',
INTERNALERROR>                                                                     'E           '
INTERNALERROR>                                                                     '}'],
INTERNALERROR>                                                           'reprfileloc': {'lineno': 11,
INTERNALERROR>                                                                           'message': 'ValidationError',
INTERNALERROR>                                                                           'path': 'test123.py'},
INTERNALERROR>                                                           'reprfuncargs': {'args': []},
INTERNALERROR>                                                           'reprlocals': None,
INTERNALERROR>                                                           'style': 'long'},
INTERNALERROR>                                                  'type': 'DatatestReprEntry'}],
INTERNALERROR>                                 'style': 'long'},
INTERNALERROR>               'sections': []},
INTERNALERROR>  'nodeid': 'test123.py::test_should_failed',
INTERNALERROR>  'outcome': 'failed',
INTERNALERROR>  'sections': [],
INTERNALERROR>  'testrun_uid': 'c913bf205a874a50a237dcf40d482d06',
INTERNALERROR>  'user_properties': [],
INTERNALERROR>  'when': 'call',
INTERNALERROR>  'worker_id': 'gw0'}
INTERNALERROR> 'Please report this bug at https://github.com/pytest-dev/pytest/issues'
INTERNALERROR> '----------------------------------------------------------------------------------------------------'
[gw0] node down: <ExceptionInfo RuntimeError('\'----------------------------------------------------------------------------------------------------\'.../issues\'\n\'----------------------------------------------------------------------------------------------------\'\n') tblen=14>
[gw0] FAILED test123.py::test_should_failed 

replacing crashed worker gw0
[gw1] darwin Python 3.8.3 cwd: /Users/qa/PycharmProjects/qa
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/_pytest/main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/xdist/dsession.py", line 112, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/xdist/dsession.py", line 135, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/xdist/dsession.py", line 263, in worker_runtest_protocol_complete
INTERNALERROR>     self.sched.mark_test_complete(node, item_index, duration)
INTERNALERROR>   File "/Users/qa/PycharmProjects/qa/venv/lib/python3.8/site-packages/xdist/scheduler/load.py", line 151, in mark_test_complete
INTERNALERROR>     self.node2pending[node].remove(item_index)
INTERNALERROR> KeyError: <WorkerController gw0>

But if I change second test like this, all works fine:

def test_should_failed():
    try:
        with accepted(Extra):
            __validate({"qwe": 1}, {"qwe": 2}, "")
    except:
        raise ValueError

I don't know exactly where i should create bug\issue about this :)

shawnbrown commented 3 years ago

Thanks for this issue. As you noted, this failure is only triggered when using the pytest-xdist command line option -n 1.

I have replicated the bug locally and will explore how to best fix this.

shawnbrown commented 3 years ago

I just released an updated version of the pytest-datatest development package (version 0.1.4). This update will be included in the next version of datatest itself.

HOW TO FIX THIS: If you want to apply this fix right now, you can install the separate plugin package (it's compatible with datatest 0.9.6):

pip install pytest-datatest

Once these changes land in the next stable version of datatest, I'll close this issue. But let's leave it open for now in case anyone else needs this same fix.

shawnbrown commented 3 years ago

With the release of versions 0.10.0 and 0.11.0, this fix is now in the stable release.