search

wemake-services / django-test-migrations

Test django schema and data migrations, including migrations' order and best practices.
https://pypi.org/project/django-test-migrations/
MIT License
527 stars 30 forks source link

Deadlock when using pants, pytest and django-test-migrations #341

Open xander-m2k opened 1 year ago

xander-m2k commented 1 year ago

I'm not really sure on what to blame on this, because it can be either pants, pytest or django-test-migrations

I'm trying to fix a problem where the migration test ends up in a deadlock, when ran using pants. I highly suspect this is due to tests running parallel (default in pants), but this can also be something you can incorporate in this library.

The standard config I have is based on this repository. As you can see in the pytest.ini, the option --reuse-db is enabled. This is for optimization purposes, just so that for every test the database does not have to be recreated. (I also think this plays a role in the ability of parallel tasks of pants, but I'm not entirely sure.)

Just because of this option, when doing a migration in a test suite, you can already guess, it causes a deadlock. When you modify tables while other tests are using the database, weird stuff happens...

Anyway, django-test-migrations runs fine when running isolated, but in a test suite is fails (and all tests coming after fail). I have a feeling that it might have something to do with this: https://stackoverflow.com/a/66732609. Is there a way to fix this deadlocking behavior?

skarzi commented 1 year ago

hi 👋

currently running Django's migrations tests with --reuse-db is not supported :( To solve your issue, you can try:

  1. Run migrations tests separately in 1 process (not parallel). So use the pytest CLI option: -m not migrations_test when running tests with pants and then pytest -m migrations_test when running migrations tests.
  2. Run both tests types without the --reuse-db flag - it should work then because pytest-django should properly populate test DB names when running together with pytest-xdist - reference - to confirm that let's check what test DBs are used when you run tests with pants