Azure / pytest-azurepipelines

Plugin for pytest that makes it simple to work with Azure Pipelines
MIT License
112 stars 35 forks source link

Incompatibility with pytest-xdist – `KeyError: 'reason'` #71

Closed joerick closed 1 year ago

joerick commented 2 years ago

When I run pytest with both pytest-azurepipelines and pytest-xdist (for parallel builds), and a test fails, I get this error:

...cut...

  File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 201, in test_suites
    return [
  File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 219, in <listcomp>
    test_case=self.test_cases(nodeid),
  File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 159, in test_cases
    return [
  File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 177, in <listcomp>
    reason=ReasonType(message=CdataComment(text=case["reason"])),
KeyError: 'reason'
full traceback ```text Traceback (most recent call last): File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/bin/pytest", line 10, in sys.exit(console_main()) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/_pytest/config/__init__.py", line 187, in console_main code = main() File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/_pytest/config/__init__.py", line 164, in main ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main( File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall return outcome.get_result() File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result raise ex[1].with_traceback(ex[2]) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall res = hook_impl.function(*args) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/_pytest/main.py", line 315, in pytest_cmdline_main return wrap_session(config, _main) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/_pytest/main.py", line 303, in wrap_session config.hook.pytest_sessionfinish( File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_callers.py", line 55, in _multicall gen.send(outcome) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/_pytest/terminal.py", line 798, in pytest_sessionfinish outcome.get_result() File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result raise ex[1].with_traceback(ex[2]) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall res = hook_impl.function(*args) File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/plugin.py", line 412, in pytest_sessionfinish result = NunitTestRun(self).generate_xml() File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 263, in generate_xml tr = self.as_test_run() File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 257, in as_test_run test_suite=self.test_suites, File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 201, in test_suites return [ File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 219, in test_case=self.test_cases(nodeid), File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 159, in test_cases return [ File "/Users/joerick/Library/Application Support/hatch/env/virtual/pytest-nunit-keyerror-iTJZd9Ui/pytest-nunit-keyerror/lib/python3.10/site-packages/pytest_nunit/nunit.py", line 177, in reason=ReasonType(message=CdataComment(text=case["reason"])), KeyError: 'reason' ```

I was able to minimally recreate like this:

# file at tests/test_fail.py

def test_fail():
    assert False
$ pip install pytest pytest-xdist pytest-azurepipelines
$ pytest -x --numprocesses=2 tests

I wasn't sure if this should be filed at pytest-nunit or here, apologies if I got it wrong!

sdebruyn commented 2 years ago

I can reproduce this with pytest-parallel

tonybaloney commented 1 year ago

This is fixed in pytest-nunit 1.0.2 I'll patch this library to use that update

tonybaloney commented 1 year ago
{'setup-report': <TestReport 'tests/test_fail.py::test_fail' when='setup' outcome='passed'>, 'call-report': <TestReport 'tests/test_fail.py::test_fail' when='call' outcome='failed'>, 'teardown-report': None, 'idref': 100, 'path': 'tests/test_fail.py', 'properties': {'python-version': '3.10.0 (v3.10.0:b494f5935c, Oct  4 2021, 14:59:20) [Clang 12.0.5 (clang-1205.0.22.11)]', 'fspath': 'tests/test_fail.py'}, 'attachments': None, 'error': '[gw1] darwin -- Python 3.10.0 /Users/anthonyshaw/tmp/pytest_repro/.venv/bin/python\n\n    def test_fail():\n>       assert False\nE       assert False\n\ntests/test_fail.py:2: AssertionError', 'stack-trace': '/Users/anthonyshaw/tmp/pytest_repro/tests/test_fail.py:2: assert False', 'name': 'tests/test_fail.py::test_fail', 'start': datetime.datetime(2022, 10, 20, 0, 47, 30, 864035)}
tonybaloney commented 1 year ago

@joerick Bug is fixed in latest pytest-nunit