Teardown of fixture with scope=class is not called if following test passes only after rerun or fails after rerun

[ExplorerOL]

Good day! I found out a problem, that teardown of fixture with scope=class not called if following test passed only after rerun or fails after rerun. I tend to think that problem is in plugin pytest-rerunfailures. I use python==3.11.0, pytest==7.2.1, pytest-rerunfailures==12.0. It also happens with pytest-rerunfailures==11.1.2. It causes big problems with my testrun.

Sample of code when teardown of fixture is not called when following test passes only after rerun:

import pytest

rerun_number = 0

@pytest.fixture(scope="class")
def class_fixture():
    print("Fixture class_fixture setup")
    yield
    print("Fixture class_fixture teardown")

class TestSuite1:
    def test_1(self, class_fixture):
        """Test 1"""
        print("Test 1")

    def test_2(self):
        """Test 1"""
        print("Test 1")

        # --- Rerun condition ---
        global rerun_number
        print(f"Rerun number = {rerun_number}")
        if rerun_number < 1:
            rerun_number += 1
            assert False
        # -----------------------

Log:

PS D:\autotests\git\tests> .\venv\Scripts\python.exe -X utf8 -m pytest -vv test_rerun_check3.py --reruns=1 --setup-show -s

test_rerun_check3.py::TestSuite1::test_1
SETUP    S base_url
SETUP    S _verify_url (fixtures used: base_url)
SETUP    S pytestconfig
SETUP    S delete_output_dir (fixtures used: pytestconfig)Fixture class_fixture setup

      SETUP    C class_fixture
        test_rerun_check3.py::TestSuite1::test_1 (fixtures used: _verify_url, base_url, class_fixture, delete_output_dir, pytestconfig, request)Test 1
PASSED
test_rerun_check3.py::TestSuite1::test_2
        test_rerun_check3.py::TestSuite1::test_2 (fixtures used: _verify_url, base_url, delete_output_dir, pytestconfig, request)Test 1
Rerun number = 0
RERUN
test_rerun_check3.py::TestSuite1::test_2
        test_rerun_check3.py::TestSuite1::test_2 (fixtures used: _verify_url, base_url, delete_output_dir, pytestconfig, request)Test 1
Rerun number = 1

TEARDOWN S delete_output_dir
TEARDOWN S pytestconfig
TEARDOWN S _verify_url
TEARDOWN S base_urlPASSED

Sample of code when teardown of fixture is not called when following test fails after rerun:

import pytest

rerun_number = 0

@pytest.fixture(scope="class")
def class_fixture():
    print("Fixture class_fixture setup")
    yield
    print("Fixture class_fixture teardown")

class TestSuite1:
    def test_1(self, class_fixture):
        """Test 1"""
        print("Test 1")

    def test_2(self):
        """Test 1"""
        print("Test 1")

        # --- Rerun condition ---
        global rerun_number
        print(f"Rerun number = {rerun_number}")
        if rerun_number < 2:
            rerun_number += 1
            assert False
        # -----------------------

Log:

PS D:\autotests\git\tests> .\venv\Scripts\python.exe -X utf8 -m pytest -vv test_rerun_check3.py --reruns=1 --setup-show -s

test_rerun_check3.py::TestSuite1::test_1
SETUP    S base_url
SETUP    S _verify_url (fixtures used: base_url)
SETUP    S pytestconfig
SETUP    S delete_output_dir (fixtures used: pytestconfig)Fixture class_fixture setup

      SETUP    C class_fixture
        test_rerun_check3.py::TestSuite1::test_1 (fixtures used: _verify_url, base_url, class_fixture, delete_output_dir, pytestconfig, request)Test 1
PASSED
test_rerun_check3.py::TestSuite1::test_2
        test_rerun_check3.py::TestSuite1::test_2 (fixtures used: _verify_url, base_url, delete_output_dir, pytestconfig, request)Test 1
Rerun number = 0
RERUN
test_rerun_check3.py::TestSuite1::test_2
        test_rerun_check3.py::TestSuite1::test_2 (fixtures used: _verify_url, base_url, delete_output_dir, pytestconfig, request)Test 1
Rerun number = 1

TEARDOWN S delete_output_dir
TEARDOWN S pytestconfig
TEARDOWN S _verify_url
TEARDOWN S base_urlFAILED

Sample of code without reruns and normal teardown:

import pytest

rerun_number = 0

@pytest.fixture(scope="class")
def class_fixture():
    print("Fixture class_fixture setup")
    yield
    print("Fixture class_fixture teardown")

class TestSuite1:
    def test_1(self, class_fixture):
        """Test 1"""
        print("Test 1")

    def test_2(self):
        """Test 1"""
        print("Test 1")

        # # --- Rerun condition ---
        # global rerun_number
        # print(f"Rerun number = {rerun_number}")
        # if rerun_number < 2:
        #     rerun_number += 1
        #     assert False
        # # -----------------------

Log without reruns:

PS D:\autotests\git\tests> .\venv\Scripts\python.exe -X utf8 -m pytest -vv test_rerun_check3.py --reruns=1 --setup-show -s

test_rerun_check3.py::TestSuite1::test_1
SETUP    S base_url
SETUP    S _verify_url (fixtures used: base_url)
SETUP    S pytestconfig
SETUP    S delete_output_dir (fixtures used: pytestconfig)Fixture class_fixture setup

      SETUP    C class_fixture
        test_rerun_check3.py::TestSuite1::test_1 (fixtures used: _verify_url, base_url, class_fixture, delete_output_dir, pytestconfig, request)Test 1
PASSED
test_rerun_check3.py::TestSuite1::test_2
        test_rerun_check3.py::TestSuite1::test_2 (fixtures used: _verify_url, base_url, delete_output_dir, pytestconfig, request)Test 1
Fixture class_fixture teardown

      TEARDOWN C class_fixture
TEARDOWN S delete_output_dir
TEARDOWN S pytestconfig
TEARDOWN S _verify_url
TEARDOWN S base_urlPASSED

Could you help me, is this problem related with this plugin?

[ExplorerOL]

In compatibilities I foud, that "This plugin may not be used with class, module, and package level fixtures"! I am really sad. I tried to modify code of plugin and managed to make it working with class-scoped fixture! My code may be not perfect and after reruns fixture with scope=class may called several times even in class, but in my case it is better than teardown is not called at all!

I modified a part of original code in the following way:

        if item in item.session._setupstate.stack:
            if PYTEST_GTE_63:
                # original code
                # for key in list(item.session._setupstate.stack.keys()):
                #     if key != item:
                #         del item.session._setupstate.stack[key]

                # my code
                keys_to_delete = list(item.session._setupstate.stack)[:-2]
                for key in keys_to_delete:
                    if key != item:
                        del item.session._setupstate.stack[key]

            else:
                for node in list(item.session._setupstate.stack):
                    if node != item:
                        item.session._setupstate.stack.remove(node)

[Mogost]

I can confirm this issue having researched one of the falls in my project.

[ExplorerOL]

@icemac Hello! As I understand the incompatibility "This plugin may not be used with class, module, and package level fixtures" is overcome now. Am I right?

[icemac]

@ExplorerOL I think this is no longer a problem.