Koed00 / django-q

A multiprocessing distributed task queue for Django
https://django-q.readthedocs.org
MIT License
1.83k stars 289 forks source link

Django q with pytest #716

Open ashleych opened 1 year ago

ashleych commented 1 year ago

Many thanks for a great package. Works excellent for my use case. I was beginning to write some tests that involve async tasks, and I realised these tests (pytests) were failing because the async job isnt looking at the right database. How does one point django-q /qcluster to look at the test database that pytest uses. im mostly using the default settings from Django. DATABASE related settings.py looks like this 'default': { 'ENGINE': 'django.db.backends.sqlite3', 'NAME': BASE_DIR / 'db.sqlite3', 'TEST': { 'NAME': BASE_DIR / 'testdb.sqlite3', }, }, I am running this to check what is the db being connected to. from django.db import connection db_name = connection.settings_dict['NAME'] When the function is being run in non async mode - it rightly points to the testdb.sqlite3. When run using async mode, it points to the db.sqlite3. How do i make the async task to refer to the testdb. Any pointers would be very helpful.

PS: If I call the async function with sync=True, then everything works fine.

NickVeld commented 1 year ago

Met this problem too. (And I do not think that the pytest is one to blame here. However, I use pytest too.) Moreover, I believe it is the same problem that cause #490

Django-Q uses default DATABASES setting but django.test.utils.setup_databases called by test runners add test_ before the configured database name. So Django-Q (which runs in separate worker/django environment) adds the tasks into the default database from the settings but Django test runner tries to read task list (and so on) from the newly created test database.

NickVeld commented 1 year ago

So it is possible to make Django-Q read task list in the test from a test database.

It is possible to override the settings file for QCluster like this: python src/manage.py qcluster --settings some_package,some_module_with_settings

Then some_module_with_settings.py located in the some_package directory must be something like this:

from .settings import *   # Imports original settings including DATABASES

# If you have some more complex setup than `DATABASES = { "default": {"ENGINE": ... , "NAME": ... , ...}}`
# you may need something smarter than the following line
DATABASES["default"]["NAME"] = "test_" + DATABASES["default']["NAME"]