Closed Fak3 closed 4 years ago
Can you confirm this happens also on pytest 5.4.1?
If yes, then a minimal reproduction would be great.
Yes, 5.4.1 too. Will try to create minimal case.
Created minimal django project: https://github.com/Fak3/testproject-824-git
Thanks for the reproduction!
It looked to me like a problem in pytest's unittest support, so I looked at changes made to that in pytest in 5.4.x. Reverting https://github.com/pytest-dev/pytest/pull/5996 "unittest: do not use TestCase.debug() with --pdb
" (and other commits in that PR) fixes the problem.
cc @blueyed @nicoddemus who worked on that PR and probably have a better idea than me.
@Fak3
Please try https://github.com/pytest-dev/pytest-django/pull/825.
The problem is that Django's (rather custom) teardown gets not called (because this is throwing an exception to get out of unittest's __call__
, so that --pdb
can be used before any teardown - although you could also just add a pdb.set_tracce()
and be done).
(It's the next round of the long-standing issue from 2016, and I still think it should have been reverted first at least instead (ref https://github.com/pytest-dev/pytest/pull/6014), until it gets done properly, e.g. via something like https://github.com/blueyed/pytest/pull/107))
pytest-django could intercept Django's _post_teardown
(https://github.com/django/django/blob/1a09708dcb2f60ad5ca0a75b8de9619356f74ff6/django/test/testcases.py#L274), and do it after the call, but that would still require to first patch pytest to not use the current approach.
Please try #825.
Yes, that PR fixes this issue
Is https://github.com/pytest-dev/pytest/issues/6947 related?
I think that the issue persist in pytest-django 3.9.0 #835
I think that the issue persist in pytest-django 3.9.0 #835
Same problem here!
@PyB1l or @hugos94, any chance of testing this with pytest's latest master
? If the problem still persists, would appreciate a minimal project to reproduce the problem (the one provided in https://github.com/pytest-dev/pytest-django/issues/824#issuecomment-598857918 is now returning 404).
For my tests i managed to isolate the issue in Django==3.0.2. For Django===3.0.5 and all the lastet version of pytest, pytest-django, i have no issues. @hugos94 could you check your versions and let us know?
For my tests i managed to isolate the issue in Django==3.0.2. For Django===3.0.5 and all the lastet version of pytest, pytest-django, i have no issues. @hugos94 could you check your versions and let us know?
I am using the following versions:
Apparently the problem only happens after making requests in views of the application. In tests that i use graphql, the error doesn't occur.
pytest 5.4.2
is out with the fix I mentioned earlier. If people can try that version and see if this is still a regression, would appreciate it. 👍
Hello,
The problem seems to have reappeared:
pip install -U pytest-django!=4.3.0
then pytest
= 626 passed, 6 skipped, 70 warnings
pip install -U pytest-django==4.3.0
then pytest
= 8 failed, 618 passed, 6 skipped, 70 warnings
Only the 2 tests below have an issue, which is weird considering the high number of tests still working fine:
@pytest.mark.parametrize('email', (
'test@aaa.fr',
'test+aaa@test.com',
'test+aaa@test.reallylongtld',
'test@abc.def.ghi.test.reallylongtldaaaaaaaaaaaaaaa',
'test@tld-minus.com',
't!#$%&es!#$%&t@tld-minus.com',
# Those are NOT valid, but we accept them to avoid an overly
# complicated regex. In any case, the true validation SHOULD be to send
# a confirmation email.
'test---@tld-minus.com',
))
@pytest.mark.django_db
def test_email_valid(self, db_setup_company: Company, email: str) -> None:
company = db_setup_company
user = User.objects.create(email=email, company=company)
assert list(user.building_ids) == []
@pytest.mark.parametrize('email', (
'aaaa@bbbb@tld-minus.com',
'@@@test@@@@tld-minus.com',
))
@pytest.mark.django_db
def test_email_invalid(self, db_setup_company: Company, email: str) -> None:
company = db_setup_company
with pytest.raises(IntegrityError) as exc:
User.objects.create(email=email, company=company)
assert 'violates check constraint "email_valid_chk"' in str(exc)
I noticed this problem as well. Do you prefer to re-open this bug or create a new issue? FWIW, here is the list of my other dependencies.
django==3.0.14
pytest-cov==2.12.0
pytest-env==0.6.2
pytest-forked==1.3.0
pytest-freezegun==0.4.2
pytest-instafail==0.4.2
pytest-mock==3.6.1
pytest-socket==0.4.0
pytest-subtests==0.4.0
pytest-vcr==1.0.2
pytest-xdist==2.2.1
pytest==6.2.4
My Django version is different from @nathanprat (3.0 vs 3.2) so it doesn't seem relevant. I see 3 common points with the above freeze.txt
:
pytest==6.2.4
pytest-django==4.3.0
pytest-mock==3.6.1
after further investigation, I've created a minimal project with out issue, you can see the failure in the action executions.
I've tracked down the problem to a custom fixture in our project. Doesn't sounds like a regression of this bug after all.
@browniebroke if you would, could you explain why that particular fixture is causing the problem? Might help others with finding similar issues.
I don't know why (yet) unfortunately, I've just managed to isolate this as the root cause, but I didn't persist further with the upgrade due to lack of time.
In case it helps, this fixture is supposed to persist model data across test cases within a class, a bit like setTestData
does for Django's TestCase
class. It's working for simple use cases, but at times, I've seen some side effects across tests, which are not present with setTestData
.
Thanks for the investigation.
The bug was indeed caused by a class-level db fixture.
I don't have time to investigate deeper right now, but the code is below if anyone is interested:
@pytest.fixture(name='class_db', scope='class')
def fixture_class_db(request: 'Any',
django_db_setup: 'Any', # pylint: disable=unused-argument
django_db_blocker: 'Any') -> None:
"""
To avoid the error with 'fixture_db_setup(db)':
"ScopeMismatch: You tried to access the 'function' scoped fixture 'db'
with a 'module' scoped request object, involved factories"
We reimplement a 'db'-like fixture class-scoped.
"""
# Copy-pasted from pytest_django.fixtures
if "django_db_reset_sequences" in request.fixturenames:
request.getfixturevalue("django_db_reset_sequences")
if ("transactional_db" in request.fixturenames
or "live_server" in request.fixturenames):
request.getfixturevalue("transactional_db")
else:
_django_db_fixture_helper(
request,
django_db_blocker,
transactional=False)
@browniebroke If you change your code from django_db_blocker.unblock()
to with django_db_blocker.unblock(): ...
does it help any?
Thanks for the suggestion @bluetech. I tried your suggestion (I think) but no luck yet... Was it the solution you had in mind?
Yep. Although it was actually also correct before, my eye just glanced over the addfinalizer
calls. Since you're now using a context manager you should be able to remove the request.addfinalizer(django_db_blocker.restore)
. You can also convert the request.addfinalizer(functools.partial(TestCase._rollback_atomics, atomics))
to a regular call after the yield
.
(I understand this doesn't actually fix your problem. What I think you are trying to achieve -- safe sharing of DB data beyond a single test -- is currently a missing feature in pytest-django)
Just for note.
In some cases it happens when you override TestCase::setUpClass()
and TestCase::tearDownClass()
methods but forget to call super-method within its. Please check you code these methods right.
I had made this mistake. After inserting super().tearDownClass()
all errors gone.
This thread helped me to note my fault: https://code.djangoproject.com/ticket/31065.
Just a quick update on my earlier issue is now resolved in pytest-django 4.5.0 and above 😄
Not sure what nor who solved it, but I just wanted to say thanks to the maintainer!
In some cases it happens when you override TestCase::setUpClass() and TestCase::tearDownClass() methods but forget to call super-method within its. Please check you code these methods right.
as an additional note: for me it happened when calling super().setUpClass()
from a overriden TestCase::setUp()
by mistake. When I removed the erroneous call the overall issue was gone.
pytest-django==3.5.1 django versions tested: 2.2.11 and 3.0.4
When one test fails, all tests which use database launched after this test will fail with traceback:
Downgrading to pytest==5.3.5 fixes the issue.
Not yet sure if this is pytest issue or pytest-django.