regression in pytest 8.1.1: module level setup function doesn't seem to be run

[neutrinoceros]

The following is a minimal reproducer for a real-life application in yt:

# test.py
obj = None
def setup():
    global obj
    obj = []

def test_foo():
    assert len(obj) == 0

This test worked as expected until pytest 8.0.2 but fails with version 8.1.1 as

=========================== test session starts ===========================
platform darwin -- Python 3.12.2, pytest-8.1.1, pluggy-1.4.0
rootdir: /private/tmp
collected 1 item

test.py F                                                           [100%]

================================ FAILURES =================================
________________________________ test_foo _________________________________

    def test_foo():
>       assert len(obj) == 0
E       TypeError: object of type 'NoneType' has no len()

test.py:7: TypeError
========================= short test summary info =========================
FAILED test.py::test_foo - TypeError: object of type 'NoneType' has no len()
============================ 1 failed in 0.01s ===========================

which seems to indicate that the setup function wasn't run ahead of the test. Possibly related to https://github.com/pytest-dev/pytest/issues/12011 or how it was resolved ?

exact env:

Package   Version
--------- -------
iniconfig 2.0.0
packaging 24.0
pluggy    1.4.0
pytest    8.1.1

Bug seen on MacOS 14 and ubuntu-latest on GHA

• [x] a detailed description of the bug or problem you are having
• [x] output of pip list from the virtual environment you are using
• [x] pytest and operating system versions
• [x] minimal example if possible

[bluetech]

setup was part of nose compatibility which is removed in pytest 8: https://docs.pytest.org/en/stable/deprecations.html#support-for-tests-written-for-nose

Use setup_module instead.

[neutrinoceros]

Oh, so the bug is that it still worked in pytest 8.0.x, is that it ?

[neutrinoceros]

I would expect that changing def setup() -> def setup_module() wouldn't change how pytest behaves at all, but this doesn't seem to be the case: I'm actually seeing a bunch of surprising failures when I do just that (see https://github.com/yt-project/yt/pull/4852/checks). Admittedly, our test suite is too complex to easily analyse and I wouldn't be surprise if what we're seeing here was a symptom of test pollution, but I'm still surprised that this wasn't visible before. Any clue ?

[bluetech]

I would expect that changing def setup() -> def setup_module() wouldn't change how pytest behaves at all

This is the expectation.

I tried to take a look but the project is quite formidable so I can't quickly figure out what's going wrong.

Is it possible that the breakage is from upgrading to pytest 8 and not necessarily from setup -> setup_module? One way to check this is to do the setup_module change with the pytest<8 pin and see what happens (setup_module is not a new thing so this is fully backward compatible).

[neutrinoceros]

I tried to take a look but the project is quite formidable so I can't quickly figure out what's going wrong.

yeah we have a lot of technical debt. Thanks for taking a shot still :)

Is it possible that the breakage is from upgrading to pytest 8 and not necessarily from setup -> setup_module? One way to check this is to do the setup_module change with the pytest<8 pin and see what happens (setup_module is not a new thing so this is fully backward compatible).

I'll try that !

[neutrinoceros]

So actually what happened is that I forgot to also migrate def teardown() -> def teardown_module(). Everything looks good now so I think we can just close this. Thanks @bluetech for your feedback !