Since upgrading to pytest>=7.0.0, the self parameter between test runs remain the same when retrying a single test function within a test class. When using pytest==6.5.2, it is working as expected, the self parameter refers to a new instance every time the test is run.
To reproduce the issue, you can use the following code:
I could work around this issue in a really ugly and fragile way using a pytest hook like this, but I think this should be fixed properly within flaky:
import pytest
import weakref
from typing import Any, Generator
last_weak_obj_key = pytest.StashKey[weakref.ref[Any]]()
last_weak_instance_key = pytest.StashKey[weakref.ref[Any]]()
@pytest.hookimpl(hookwrapper=True, tryfirst=True)
def pytest_runtest_setup(item: Any) -> Generator[None, Any, None]:
last_weak_obj = item.stash.get(last_weak_obj_key, lambda: None)
last_weak_instance = item.stash.get(last_weak_instance_key, lambda: None)
assert item.obj is not None
if item.obj is last_weak_obj() or (
item.instance is not None and item.instance is last_weak_instance()
):
# Deleting these two internal attributes trigger new instance creation when accessing item.obj or item.instance the next time
del item._obj
del item._instance
assert item.obj is not None and item.obj is not last_weak_obj()
item.stash[last_weak_obj_key] = weakref.ref(item.obj)
assert item.instance is None or item.instance is not last_weak_instance()
item.stash[last_weak_instance_key] = weakref.ref(item.instance)
yield
Since upgrading to
pytest>=7.0.0
, theself
parameter between test runs remain the same when retrying a single test function within a test class. When usingpytest==6.5.2
, it is working as expected, theself
parameter refers to a new instance every time the test is run.To reproduce the issue, you can use the following code:
Running these tests output the following, showing that the same instance is used for
self
when retrying the test:I think this should be considered as a bug, see for example the commit message of https://github.com/pytest-dev/pytest/commit/0dc036035107b213c9b73bf965cbd7356111b85a which states
This is definitely broken since it breaks test isolation.
for a similar case.I could work around this issue in a really ugly and fragile way using a pytest hook like this, but I think this should be fixed properly within
flaky
:As
pytest-rerunfailures
is also affected, I've opened this issue there as well: https://github.com/pytest-dev/pytest-rerunfailures/issues/268