Open SpoonMeiser opened 5 years ago
With further investigation, I now think this problem also exists when using version 1.26.0. That was the only thing I could find that had changed that might be related to this problem, so now I'm no longer sure if this problem is directly related to xdist or not.
For the time being, we've stopped using xdist altogether, and the problem doesn't occur. I'm happy for this to be closed and I can raise again if I investigate further in the future and narrow down the problem better, unless anyone thinks they could make any progress with the information I have provided so far.
there seems to be more than 10 still existing users of the database, i suspect that xdist is not related, but makes the issue so much worse that it happens every time
pytest xdists should make a separate database (IIUC) for each worker, so there should only be one user of each database. I don't understand what situation could occur that would increase that number.
It seems as though the number of users of the database is loosely related to the number of workers (it differs on each run)
@SpoonMeiser its possibly related to the pytest upgrade where we moved xunit setup/teardown into the fixture system, i recall pytest-django replicating part o that, thus being triggered into incorrect behavior, i suggest testing against a lower pytest version just to verify that idea
Thanks, it's super useful to just get another suggestion of something to test, I'll test that and get back to you.
@SpoonMeiser its possibly related to the pytest upgrade where we moved xunit setup/teardown into the fixture system, i recall pytest-django replicating part o that, thus being triggered into incorrect behavior, i suggest testing against a lower pytest version just to verify that idea
Yeah, pytest-django skips its handling of unittests with pytest 4.2+, but there appear to be issues with that (i.e. it is not really compatible / the same).
Are you using unittest / Django's TestCase etc?
Any update on this?
Pretty sure I'm looking at the same bug here, so adding my notes. We have a very db-intensive test suite (needs to be for this app). We use Postgres, pytest, pytest-django, and pytest-xdist on MacOS. Previously, we could run our test suite with -n 8
and the parallelism would work just fine; 8 databases were created and destroyed. But lately we've been seeing a lot of errors like the one described here. Specifically:
self.log('Got an error creating the test database: %s' % e)
> sys.exit(2)
E SystemExit: 2
I've done some testing and come up with the example below, in case it's helpful debugging this.
Decorating any test with @pytest.mark.django_db
invokes db creation, even if no data is created or touched in the test body. Note that this sample test does not use any code from our project -- it's "bare":
# sometest.py
import pytest
@pytest.mark.django_db
def test_a():
assert 1 == 1
@pytest.mark.django_db
def test_b():
assert 1 == 1
pytest sometest.py
it works just fine.pytest -n 2 sometest.py
it fails with a database creation error similar to the one above.pytest -n 2 sometest.py
it's fine again.In other words, the problem only comes up when xdist
is invoked AND the pytest collector finds more than one test that touches the database.
Current testrunner libs are:
plugins: testinfra-1.16.0, sugar-0.9.4, html-3.1.1, cov-2.11.1, metadata-1.11.0, xdist-2.2.1, faker-2.0.0, Faker-4.1.2, django-4.1.0, flakefinder-1.0.0, forked-1.3.0
Going on other suggestions, I've tried downgrading python-xdist to 1.34.0 but it did not fix the problem.
It is not clear to me whether the problem is in pytest-django or in python-xdist. I'm going to cross-post this to issues on both projects.
I seem to have this issue as well
In my case, it was caused by the wrong config for databases:
DATABASES = {
'default': {
'NAME': 'postgres',
},
'caching_database': {
'NAME': 'postgres',
},
'user_db': {
'NAME': 'postgres',
}
}
So pytest-xdist
tried to create a new database using the database name test_postgres_gw0
three times and as a result, failed with SystemExit: 2
pytest with pytest-django is giving us errors tearing down databases since upgrading to pytest-xdist 1.26.1. This happens nearly 100% of the time, but very occasionally doesn't. I suspect that having a large number of tests exacerbates the problem.
We get an error like this (anonymised):
I have been running with pytest-random and the
--random
argument to test this behaviour, and the name of the test that failed is different every time, and I don't think relevant to this behaviour; I don't think this is a problem with any of the tests. This problem occurs without the pytest-random plugin installed - I was just using that to convince myself that the problem wasn't related to a particular test suite.Sometimes an additional failure with as IntegrityError will occur on the primary key of a record when inserting into the database, but I think this a symptom of the above error rather than a legitimate test error.
The commandline used is:
pytest --ds=test_settings_local --disable-warnings -n auto --random
This problem started occurring around the time pytest-xdist 1.26.1 was released, and downgrading to pytest-xdist 1.26.0 makes the problem (seem to) go away, which leads me to believe that the problem is with pytest-xdist 1.26.1