search

pytest-dev / pytest-rerunfailures

a pytest plugin that re-runs failed tests up to -n times to eliminate flakey failures
Other
369 stars 82 forks source link

Tests are terminating with internal error #235

Open vivekvasu95 opened 10 months ago

vivekvasu95 commented 10 months ago 
14:02:03  INTERNALERROR> def worker_internal_error(self, node, formatted_error):
14:02:03  INTERNALERROR>         """
14:02:03  INTERNALERROR>         pytest_internalerror() was called on the worker.
14:02:03  INTERNALERROR>     
14:02:03  INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr, which can't
14:02:03  INTERNALERROR>         be serialized, so we go with a poor man's solution of raising an exception
14:02:03  INTERNALERROR>         here ourselves using the formatted message.
14:02:03  INTERNALERROR>         """
14:02:03  INTERNALERROR>         self._active_nodes.remove(node)
14:02:03  INTERNALERROR>         try:
14:02:03  INTERNALERROR> >           assert False, formatted_error
14:02:03  INTERNALERROR> E           AssertionError: Traceback (most recent call last):
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/_pytest/main.py", line 270, in wrap_session
14:02:03  INTERNALERROR> E                 session.exitstatus = doit(config, session) or 0
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/_pytest/main.py", line 324, in _main
14:02:03  INTERNALERROR> E                 config.hook.pytest_runtestloop(session=session)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__
14:02:03  INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec
14:02:03  INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 155, in _multicall
14:02:03  INTERNALERROR> E                 return outcome.get_result()
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_result.py", line 108, in get_result
14:02:03  INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall
14:02:03  INTERNALERROR> E                 res = hook_impl.function(*args)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/xdist/remote.py", line 146, in pytest_runtestloop
14:02:03  INTERNALERROR> E                 self.run_one_test()
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/xdist/remote.py", line 161, in run_one_test
14:02:03  INTERNALERROR> E                 self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__
14:02:03  INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec
14:02:03  INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 155, in _multicall
14:02:03  INTERNALERROR> E                 return outcome.get_result()
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_result.py", line 108, in get_result
14:02:03  INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall
14:02:03  INTERNALERROR> E                 res = hook_impl.function(*args)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pytest_rerunfailures.py", line 582, in pytest_runtest_protocol
14:02:03  INTERNALERROR> E                 if _should_not_rerun(item, report, reruns):
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pytest_rerunfailures.py", line 336, in _should_not_rerun
14:02:03  INTERNALERROR> E                 is_terminal_error = _should_hard_fail_on_error(item.session.config, report)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pytest_rerunfailures.py", line 327, in _should_hard_fail_on_error
14:02:03  INTERNALERROR> E                 if not re.search(rerun_regex, report.longrepr.reprcrash.message):
14:02:03  INTERNALERROR> E             AttributeError: 'NoneType' object has no attribute 'message'
14:02:03  INTERNALERROR> E           assert False
RonnyPfannschmidt commented 10 months ago

i took the liberty to edit for readability

based on the traceback its a issue with pytest_rerunfailures

RonnyPfannschmidt commented 10 months ago

im unfamiliar with rerunfailures, so its not clear to me if this is a bug with rerunfailures itself or something else interacting with it

icemac commented 10 months ago

@vivekvasu95 Could you please provide some version information? (version of pytest, pytest-rerunfailures, other plugins, Python, …) What did you already do to circumvent the problem? Just pasting a traceback is not enough as you seem to be the only one with this problem.

vivekvasu95 commented 10 months ago

@icemac Here is the complete packages used

Package Version

adbutils 1.0.8 allure-pytest 2.9.44 allure-python-commons 2.9.44 apkutils2 1.0.0 appdirs 1.4.4 Appium-Python-Client 2.7.1 astroid 2.15.2 async-generator 1.10 attrs 22.1.0 audioread 3.0.0 beautifulsoup4 4.11.2 boto3 1.28.11 botocore 1.31.11 Brotli 1.0.9 cached-property 1.5.2 certifi 2022.6.15 cffi 1.15.1 chainmap 1.0.3 chardet 3.0.4 charset-normalizer 2.1.1 cigam 0.0.3 click 8.1.3 combomethod 1.0.12 ConfigArgParse 1.5.3 configparser 5.3.0 contourpy 1.0.5 cryptography 37.0.4 curlify 2.2.1 cycler 0.11.0 decorator 5.1.1 Deprecated 1.2.13 deprecation 2.1.0 dill 0.3.6 emoji 2.2.0 et-xmlfile 1.1.0 exceptiongroup 1.1.1 execnet 1.9.0 Faker 18.3.0 ffmpeg-python 0.2.0 filelock 3.9.1 Flask 2.2.3 Flask-BasicAuth 0.2.0 Flask-Cors 3.0.10 fonttools 4.37.4 future 0.18.2 geographiclib 2.0 geopy 2.3.0 gevent 21.12.0 geventhttpclient 2.0 greenlet 2.0.1 h11 0.13.0 html2text 2020.1.16 hvac 1.1.1 idna 2.10 influxdb 5.3.1 influxdb-client 1.37.0 iniconfig 1.1.1 isort 5.10.1 itsdangerous 2.1.2 Jinja2 3.0.3 jmespath 1.0.1 joblib 1.2.0 kiwisolver 1.4.4 lazy-object-proxy 1.7.1 lc 0.1 librosa 0.9.2 llvmlite 0.39.1 locust 2.8.6 logzero 1.7.0 lxml 4.9.1 mailosaur 7.14.0 MarkupSafe 2.1.1 matplotlib 3.6.1 mccabe 0.6.1 msgpack 1.0.4 names 0.3.0 noisereduce 2.0.1 nulltype 2.3.1 numba 0.56.3 numpy 1.24.2 openpyxl 3.1.1 options 1.4.10 outcome 1.2.0 packaging 20.9 pandas 1.5.3 Pillow 9.2.0 pip 21.3.1 pixelmatch 0.3.0 platformdirs 2.5.2 playwright 1.28.0 pluggy 1.0.0 pooch 1.6.0 progress 1.6 psutil 5.9.1 py 1.11.0 pycparser 2.21 pydub 0.25.1 pyee 9.0.4 pyelftools 0.29 pyhcl 0.4.4 PyJWT 2.6.0 pylint 2.16.2 pyOpenSSL 22.0.0 pyparsing 3.0.9 PySocks 1.7.1 pytest 7.2.2 pytest-base-url 2.0.0 pytest-check 2.1.4 pytest-forked 1.4.0 pytest-json-report 1.5.0 pytest-metadata 2.0.4 pytest-parallel 0.1.1 pytest-playwright 0.3.0 pytest-playwright-visual 2.1.2 pytest-rerunfailures 11.1.1 pytest-timeout 2.1.0 pytest-xdist 3.2.0 python-dateutil 2.8.2 python-slugify 6.1.2 pytz 2022.7.1 pytz-deprecation-shim 0.1.0.post0 pyzmq 23.2.0 ratelimiter 1.2.0.post0 reactivex 4.0.4 requests 2.28.1 resampy 0.4.2 retry 0.9.2 roundrobin 0.0.2 s3transfer 0.6.1 scikit-learn 1.1.2 scipy 1.9.2 selenium 4.8.3 setuptools 65.6.3 simplejson 3.17.6 six 1.12.0 sniffio 1.2.0 sortedcontainers 2.4.0 soundfile 0.11.0 soupsieve 2.3.2.post1 SpeechRecognition 3.8.1 tblib 1.7.0 text-unidecode 1.3 threadpoolctl 3.1.0 toml 0.10.2 tomli 2.0.1 tomlkit 0.11.6 tqdm 4.64.1 trio 0.21.0 trio-websocket 0.9.2 typing_extensions 4.3.0 tzdata 2022.1 tzlocal 4.2 uiautomator2 2.16.19 urllib3 1.26.12 websockets 10.1 Werkzeug 2.2.2 wheel 0.37.1 whichcraft 0.6.1 wrapt 1.13.3 wsproto 1.1.0 xmltodict 0.13.0 xvfbwrapper 0.2.9 zope.event 4.5.0 zope.interface 5.4.0

icemac commented 10 months ago

@vivekvasu95 There are newer versions of both pytest and pytest-rerunfailures. Could you please try that versions as a first mitigation step?

vivekvasu95 commented 10 months ago

Sure.

icemac commented 9 months ago

I am closing this old issue. Feel free to re-open it if there are any news about the topic.