Getting a `AttributeError: _instance` when running tests on CircleCI

[MayankJ1403]

Hey @bluetech, Been following some of the changelogs and have seen a couple of your changes go thru for Pytest and pytest-django. I am trying to upgrade my Pytest version from 8.1.1 to 8.3.3 and my pytest-django version from 4.8.0 to 4.9.0.

When I'm running the tests locally, everything works as expected but on CircleCI, I am getting a rather weird stack trace

Traceback (most recent call last):
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/threadexception.py", line 97, in pytest_runtest_teardown
    yield from thread_exception_runtest_hook()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 100, in pytest_runtest_teardown
    yield from unraisable_exception_runtest_hook()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/logging.py", line 853, in pytest_runtest_teardown
    yield from self._runtest_for(item, "teardown")
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/capture.py", line 885, in pytest_runtest_teardown
    return (yield)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 189, in pytest_runtest_teardown
    item.session._setupstate.teardown_exact(nextitem)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 557, in teardown_exact
    raise exceptions[0]
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 546, in teardown_exact
    fin()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/unittest.py", line 226, in teardown
    del self._instance
AttributeError: _instance

I was hoping you may have some idea around this because i'd seen some of the discussion threads you were involved in regarding the unittest changes in 8.2.0 and the latest release for pytest-django 4.9.0

The versions used locally AND CircleCI are pytest - v8.3.3 pytest-django v4.9.0 python 3.9.18 postgres 12.18 Django - 4.2.15 django-tenant-schemas 1.12.0

The earlier versions where the pipeline was running fine was pytest - v8.1.1. pytest-django v4.8.0 python 3.9.18 postgres 12.18 Django - 4.2.15 django-tenant-schemas 1.12.0

[bluetech]

@MayankJ1403 it might be a bug but I'm afraid I'll need some more information. Are you able to narrow it down to a specific test? Does the test do anything unusual? Are there any other pytest plugins in use except pytest-django?

I'd say this error can occur in one of two ways:

• self._instance wasn't initialized but teardown ran; this in turn (if I'm analyzing possible causes) is likely because setup raises in some unexpected place.
• teardown runs twice, without setup in-between.

I'm guessing it's the first one. If you can isolate where it fails that would be most helpful.

[MayankJ1403]

I can provide a "scrubbed" variant of the test since the code is private. For context, we are using django-tenant-schemas as well which has largely remained unmainted for a long time. it could be the case that Pytest v8.2+ has some breakages for DTS specifically.

class ViewSetTest(FastTenantAPITestCase):
    PATCH_ENDPOINT = "api:endpoint"

    @classmethod
    def setUpClass(cls):
        super().setUpClass()
        ... # FactoryBoyCode

    def setUp(self):
        self.user = UserFactory()

    def test_patch_endpoint(self):
        with self.assertNumQueriesMinusSchema(36):
            self.call_patch_endpoint(user=self.user)

from tenant_schemas.test.cases import FastTenantTestCase
class FastTenantAPITestCase(FastTenantTestCase):  

    @classmethod
    def setUpClass(cls):
        super().setUpClass()
        TestCase.setUpClass()

        # Access lazy-loaded constance configs to avoid extra queries in test cases
        constance_config.COOKIE_EXPIRY_IN_SECONDS

    @classmethod
    def tearDownClass(cls):
        super().tearDownClass()
        TestCase.tearDownClass()
        translation.activate(settings.LANGUAGE_CODE)

    def _pre_setup(self):
        super()._pre_setup()
        self.client_class = APITestClient
        self.client = self.client_class(tenant=self.tenant)

    ... # Other helper methods such as asser list equal etc.

from tenant_schemas.test.client import TenantClient
from rest_framework.test import APIClient

   class APITestClient(TenantClient, APIClient):
    pass

Whats strange is that when I run this test specifically locally or if I run the collection of tests from the specific CircleCI container locally, it works just fine. But it breaks on CircleCI.

I've tried to debug if there are any environmental differences between the two and they are identical in config.

Another thing I've noticed is that if I upgrade to v8.2.2, I face a different error

Traceback (most recent call last):
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 341, in from_call
    result: Optional[TResult] = func()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 241, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/threadexception.py", line 87, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/threadexception.py", line 63, in thread_exception_runtest_hook
    yield
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 90, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/unraisableexception.py", line 65, in unraisable_exception_runtest_hook
    yield
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/logging.py", line 833, in _runtest_for
    yield
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/capture.py", line 878, in pytest_runtest_call
    return (yield)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 183, in pytest_runtest_call
    raise e
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/_pytest/runner.py", line 173, in pytest_runtest_call
    item.runtest()
  File "/home/circleci/repo/venv/lib/python3.9/site-packages/pytest_django/plugin.py", line 553, in non_debugging_runtest
    self._testcase(result=self)
TypeError: 'NoneType' object is not callable

The plugins I am using are ( From the pytest output )

plugins: django-constance-2.9.1, ddtrace-2.9.2, celery-0.0.0, cov-5.0.0, Faker-26.0.0, django-4.8.0, forked-1.6.0, xdist-3.6.1, requests-mock-1.9.3, order-1.2.1, rerunfailures-11.0

Another thing too is that this a flaky nature of test cases. Sometimes it will pass and sometimes it will fail. I understand that this kind of flakiness can largely be caused due to test ordering which points to some incorrect handling of data from one test class to another that may affect this but I was just curious if there are any obvious blockers here that I can identify

[bluetech]

Are you able to reproduce without rerunfailures?

(BTW, you probably aren't using pytest-forked and can remove it).

[MayankJ1403]

We are using rerunfailures on CI with --reruns 3 but locally I haven't used it so I've been a little stuck on reproducing this because it just passes. Also, yeah good eye on forked. I should remove it

[The-Compiler]

Try using it then? If you can reproduce on CI but not locally, and you use rerunfailures on CI but not locally, it seems like a good next step to try getting your local configuration closer to what you do on CI?

[MayankJ1403]

I tried this. Since we were using rerunfailures-11.0 which doesn't officially support pytest8+ I figured that may be the issue so I upgraded that to the latest version of 14.0 but still ran into the issue. However, using rerunfailures locally did reproduce the issue for me so perhaps you are correct about your assumption that it may be due to rerunfailures

The changelogs for rerunfailures doesn't explicitly mention support for v8.2 hence it might not have incorporated any required changes to match the breaking changes of v8.2. But that's just my conjecture

[MayankJ1403]

I did find an ongoing thread for rerunfailures as well which might justify this finding

[MayankJ1403]

Closing this issue at this time because its likely a compatibility issue w/ rerunfailures. Thanks for your help @bluetech @The-Compiler Really appreciate your promptness. And love your work too!