AttributeError: 'list' object has no attribute 'items' in `_pytest.runner.SetupState.setup`

[pawamoy]

Upon running my test suite, I sometimes get these errors:

pytest -c config/pytest.ini -n auto  tests

============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.2, pluggy-1.3.0
Using --randomly-seed=196610002
rootdir: /media/data/dev/aria2p/config
configfile: pytest.ini
plugins: cov-4.1.0, xdist-3.3.1, anyio-3.7.1, rerunfailures-9.1.1, randomly-3.15.0
created: 6/6 workers
6 workers [480 items]

......................................................................... [ 15%]
........................................................................ [ 30%]
...........................................................x............ [ 45%]
........................................................................ [ 60%]
........................................................................ [ 75%]
........................................................................ [ 90%]
..............................................R                          [100%]R [100%]R [100%]R [100%]R [100%]E [100%]Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=0 mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='utf-8'>
Exception ignored in: <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name=1 mode='w' encoding='utf-8'>

==================================== ERRORS ====================================
___________________ ERROR at setup of test_pause_subcommand ____________________
[gw2] linux -- Python 3.11.5 /usr/bin/python3.11

cls = <class '_pytest.runner.CallInfo'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x7f8f124a5e40>
when = 'setup'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: "Callable[[], TResult]",
        when: "Literal['collect', 'setup', 'call', 'teardown']",
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.

        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

__pypackages__/3.11/lib/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
__pypackages__/3.11/lib/_pytest/runner.py:262: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
__pypackages__/3.11/lib/pluggy/_hooks.py:493: in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
__pypackages__/3.11/lib/pluggy/_manager.py:115: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
__pypackages__/3.11/lib/_pytest/runner.py:157: in pytest_runtest_setup
    item.session._setupstate.setup(item)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_pytest.runner.SetupState object at 0x7f8f19b3a6d0>
item = <Function test_pause_subcommand>

    def setup(self, item: Item) -> None:
        """Setup objects along the collector chain to the item."""
        needed_collectors = item.listchain()

        # If a collector fails its setup, fail its entire subtree of items.
        # The setup is not retried for each item - the same exception is used.
>       for col, (finalizers, exc) in self.stack.items():
E       AttributeError: 'list' object has no attribute 'items'

__pypackages__/3.11/lib/_pytest/runner.py:484: AttributeError
----------------------------- Captured stderr call -----------------------------
GID#0000000000000001 cannot be paused now
=========================== short test summary info ============================
ERROR config/test_cli.py::test_pause_subcommand - AttributeError: 'list' object has no attribute 'items'
============== 478 passed, 1 xfailed, 1 error, 5 rerun in 20.20s ===============

The error happens on random tests (not always the same ones), sometimes on a single test, sometimes on many more.

My repo: https://github.com/pawamoy/aria2p. My config:

[pytest]
norecursedirs =
  .git
  .tox
  .env
  dist
  build
python_files =
  test_*.py
  *_test.py
  tests.py
addopts =
  --cov
  --cov-append
  --cov-config config/coverage.ini
  --randomly-dont-reset-seed
  --reruns 5
  --reruns-delay 0.1
testpaths =
  tests

# action:message_regex:warning_class:module_regex:line
filterwarnings =
  error
  # TODO: remove once pytest-xdist 4 is released
  ignore:.*rsyncdir:DeprecationWarning:xdist
  ignore:.*the imp module:DeprecationWarning:future

Operating system: Linux. Pytest and plugins:

pytest-cov==4.1.0
pytest-randomly==3.15.0
pytest-rerunfailures==9.1.1
pytest-xdist==3.3.1
pytest==7.4.2

All packages

``` -e file:///media/data/dev/aria2p#egg=aria2p annotated-types==0.6.0 ansimarkup==1.5.0 anyio==3.7.1 appdirs==1.4.4 asciimatics==1.14.0 Babel==2.13.0 beautifulsoup4==4.12.2 black==23.9.1 blacken-docs==1.16.0 certifi==2023.7.22 charset-normalizer==3.3.0 click==8.1.7 colorama==0.4.6 coverage==7.3.2 csscompressor==0.9.5 dparse==0.6.3 duty==1.0.0 editables==0.5 execnet==2.0.2 failprint==1.0.2 fastapi==0.103.2 future==0.18.3 ghp-import==2.1.0 git-changelog==2.3.1 gitdb==4.0.10 GitPython==3.1.37 griffe==0.36.5 h11==0.14.0 htmlmin2==0.1.13 idna==3.4 iniconfig==2.0.0 Jinja2==3.1.2 jsmin==3.0.1 loguru==0.7.2 lxml==4.9.3 markdown-callouts==0.3.0 markdown-exec==1.6.0.1.0.1 Markdown==3.5 MarkupSafe==2.1.3 mergedeep==1.3.4 mkdocs-autorefs==0.5.0 mkdocs-coverage==1.0.0 mkdocs-gen-files==0.5.0 mkdocs-git-committers-plugin-2==1.2.0 mkdocs-literate-nav==0.6.1 mkdocs-material-extensions==1.2 mkdocs-material==9.4.5 mkdocs-minify-plugin==0.7.1 mkdocs==1.5.3 mkdocstrings-python==1.7.3.1.5.1 mkdocstrings==0.23.0 mypy-extensions==1.0.0 mypy==1.5.1 packaging==23.2 paginate==0.5.6 pathspec==0.11.2 Pillow==10.0.1 platformdirs==3.11.0 pluggy==1.3.0 psutil==5.9.5 ptyprocess==0.7.0 pydantic-core==2.10.1 pydantic==2.4.2 pyfiglet==0.8.post1 Pygments==2.16.1 pymdown-extensions==10.3 pyperclip==1.8.2 pytest-cov==4.1.0 pytest-randomly==3.15.0 pytest-rerunfailures==9.1.1 pytest-xdist==3.3.1 pytest==7.4.2 python-dateutil==2.8.2 pyyaml-env-tag==0.1 PyYAML==6.0.1 regex==2023.10.3 requests==2.31.0 responses==0.23.3 ruamel-yaml-clib==0.2.8 ruamel-yaml==0.17.35 ruff==0.0.292 safety==2.3.4 semver==3.0.2 setuptools==68.2.2 six==1.16.0 smmap==5.0.1 sniffio==1.3.0 soupsieve==2.5 starlette==0.27.0 toml==0.10.2 types-Markdown==3.5.0.0 types-PyYAML==6.0.12.12 types-requests==2.31.0.8 types-setuptools==68.2.0.0 types-toml==0.10.8.7 typing-extensions==4.8.0 urllib3==2.0.6 uvicorn==0.23.2 watchdog==3.0.0 wcwidth==0.2.8 websocket-client==1.6.4 ```

---

I tried disabling/enabling plugins to see from which it could come, and it seems to only happen when rerunfailures is enabled. Besides, I see code in it that could explain the error:

def _remove_failed_setup_state_from_session(item):
    """
    Note: remove all _prepare_exc attribute from every col in stack of
          _setupstate and cleaning the stack itself
    """
    prepare_exc = "_prepare_exc"
    setup_state = getattr(item.session, "_setupstate")
    for col in setup_state.stack:
        if hasattr(col, prepare_exc):
            delattr(col, prepare_exc)
    setup_state.stack = list()         # <------ here

The error seems to happen when a test fails and therefore the rerun machinery kicks in.

[ShuGuangTR]

Hi @pawamoy , I had the same problem, but I solved it. I rolled back pytest 7.4.2---->5.2.1 , and then, is worked! If you need reruns, pls try.

[icemac]

Maybe there way a change in pytest, which we do not trigger in our own tests. I'd be happy to see a PR to fix this issue. I am not sure whether it could be as easy as replacing the list with a dict.

[ShuGuangTR]

Maybe there way a change in pytest, which we do not trigger in our own tests. I'd be happy to see a PR to fix this issue. I am not sure whether it could be as easy as replacing the list with a dict.

Thanks for your answer, that's right, I saw it's fixed on https://github.com/pytest-dev/pytest-rerunfailures/pull/245/commits/3acc7b39e9e1840ec1ecf864a693cf6fac5e5bef Upgrading the pytest-rerunfailures to last version can solve this issue

=========== in pytest-rerunfailures #245 commit =======================================

    --setup_state.stack = []
    ++setup_state.stack = {}

[pawamoy]

Great, closing then! Thanks everyone!