search

pytest-dev / pytest-github-actions-annotate-failures

Pytest plugin to annotate failed tests with a workflow command for GitHub Actions
https://pypi.org/project/pytest-github-actions-annotate-failures/
MIT License
132 stars 9 forks source link

0.1.4 breaks CI: AttributeError: 'Function' object has no attribute 'execution_count' #42

Closed frenck closed 2 years ago

frenck commented 2 years ago

0.1.4 breaks our test suite on GitHub actions:

Arguments: ('Future exception was never retrieved',)
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/xdist/remote.py", line 87, in pytest_runtestloop
INTERNALERROR> E                 self.run_one_test(torun)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/xdist/remote.py", line 106, in run_one_test
INTERNALERROR> E                 self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR> E                 return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR> E                 self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/_pytest/runner.py", line 109, in pytest_runtest_protocol
INTERNALERROR> E                 runtestprotocol(item, nextitem=nextitem)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/_pytest/runner.py", line 120, in runtestprotocol
INTERNALERROR> E                 rep = call_and_report(item, "setup", log)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/_pytest/runner.py", line 217, in call_and_report
INTERNALERROR> E                 report: TestReport = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR> E                 return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR> E                 self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR> E                 gen.send(outcome)
INTERNALERROR> E               File "/__w/core/core/venv/lib/python3.8/site-packages/pytest_github_actions_annotate_failures/plugin.py", line 34, in pytest_runtest_makereport
INTERNALERROR> E                 if item.execution_count <= pytest_rerunfailures.get_reruns_count(item):
INTERNALERROR> E             AttributeError: 'Function' object has no attribute 'execution_count'
INTERNALERROR> E           assert False

Ref: https://github.com/home-assistant/core/pull/58351

marvin-w commented 2 years ago

https://github.com/utgwkk/pytest-github-actions-annotate-failures/pull/40 (well the release of it) caused the pipelines to fail.

It looks like the import check does not work correctly. Home Assistant does not use the pytest-rerunfailures plugin (at least it's not defined in any requirement and is also not manually installed during GH Action execution.

Maybe checking if execution_count exists on item is a better approach to handle this (and also saves the dirty pass of the ImportError).

execution_count is set here: https://github.com/pytest-dev/pytest-rerunfailures/blob/master/pytest_rerunfailures.py#L487

Also, one could argue that its better to fix the flaky tests instead of rerunning them in CI - but I guess that's a philosophy question.

henryiii commented 2 years ago

You should be able to check for the entry point to detect the plugin, that's how PyTest discovers plugins. Worth checking, anyway.

utgwkk commented 2 years ago

I've just released version 0.1.5 (reverted changes of 0.1.4).

billyvg commented 2 years ago

Ah sorry about this - I had manually tested this and it seemed to work, will look into a different method of detecting the plugin.

utgwkk commented 2 years ago

Closing this issue because CI error was resolved by reverting changes.