search

str0zzapreti / pytest-retry

A simple plugin for retrying flaky tests in CI environments
MIT License
27 stars 6 forks source link

fails with latest version of pytest on Python 3.11 #17

Closed tswast closed 10 months ago

tswast commented 10 months ago

With the most recent release of pytest-retry, we are seeing some unexpected failures with Python 3.11 and the latest release of pytest.

nox > py.test --quiet -n=20 --timeout=900 --junitxml=system_3.11_sponge_log.xml --cov=bigframes --cov=tests/system/small --cov-append --cov-config=.coveragerc --cov-report=term-missing --cov-fail-under=0 tests/system/small
Traceback (most recent call last):
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/bin/py.test", line 8, in <module>
    sys.exit(console_main())
             ^^^^^^^^^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/config/__init__.py", line 189, in console_main
    code = main()
           ^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/config/__init__.py", line 147, in main
    config = _prepareconfig(args, plugins)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/config/__init__.py", line 328, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pluggy/_hooks.py", line 433, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pluggy/_manager.py", line 112, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pluggy/_callers.py", line 133, in _multicall
    teardown[0].send(outcome)
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
    config: Config = outcome.get_result()
                     ^^^^^^^^^^^^^^^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pluggy/_result.py", line 108, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pluggy/_callers.py", line 80, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1075, in pytest_cmdline_parse
    self.parse(args)
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1425, in parse
    self._preparse(args, addopts=addopts)
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1305, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pluggy/_manager.py", line 364, in load_setuptools_entrypoints
    plugin = ep.load()
             ^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 178, in exec_module
    exec(co, module.__dict__)
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pytest_retry/__init__.py", line 1, in <module>
    from .retry_plugin import duration_key, outcome_key, attempts_key
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 178, in exec_module
    exec(co, module.__dict__)
  File "/tmpfs/src/git/bigframes/.nox/system-3-11/lib/python3.11/site-packages/pytest_retry/retry_plugin.py", line 8, in <module>
    from configs import Defaults
ModuleNotFoundError: No module named 'configs'
nox > Command py.test --quiet -n=20 --timeout=900 --junitxml=system_3.11_sponge_log.xml --cov=bigframes --cov=tests/system/small --cov-append --cov-config=.coveragerc --cov-report=term-missing --cov-fail-under=0 tests/system/small failed with exit code 1
nox > Session system-3.11 failed.
tswast commented 10 months ago

It might just be that my Python 3.11 environment was the one that had a conflict with the root configs.py file, but I also add some tests for Python 3.11 in my PR: https://github.com/str0zzapreti/pytest-retry/pull/18

savonarola commented 10 months ago

We have the same issue, our CI pipelines using pytest-retry (e.g. https://github.com/emqx/emqx/actions/runs/5951459235/job/16147608045?pr=11481) started to fail with the same error: ModuleNotFoundError: No module named 'configs'.

We had to pin to 1.3.0.

str0zzapreti commented 10 months ago

Aha, I can already see how this might've happened, but I'll need to do a bit of testing on my end. Thanks for filing the issue, I'll take a look at your PR in just a moment here.