Open Guest007 opened 8 months ago
you mean it is impossible to have periodic tasks stored in the database, on a per-tenant basis?
You can, but only from public
will work as expected
Do you have the periodic tasks inserted dynamically into the DB, or are they sourced from your source code?
It was tested with tasks, described in model PeriodicTask
(part of django-celery-beat
). In that way, task must be defined in source code, but conditions of use - in PeriodicTask
.
As I see, this is common scenario for manipulate task's schedule in terms of DatabaseSchedule (part of django-celery-beat
) https://django-celery-beat.readthedocs.io/
I believe the tenant_schemas_celery.scheduler.TenantAwareScheduler
scheduler class does what you need. When you don't specify the tenant_schemas
key, or set it to None
, the task will be sent to all the tenants. Am I missing something?
Yes. You missed django-celery-beat - Celery Periodic Tasks backed by the Django ORM (as wrote in About section). It's simple and clear. Except one - celery-beat don't know about tenants.
And if my clients = tenants - I have a way to make manipulation with task's schedule through admin individually by client.
One client want to run, for example, some data imports each day at 3, but other - only weekly. And third client don't use this task at all.
Project https://github.com/QuickRelease/django-tenants-celery-beat collects all tasks from all tenants to one model, placed in public
and run tasks from that place. I guess it overhead.
celery-beat-tenants-scheduler
just run through all tenants to work.
Since you are adding the tasks dynamically to the DB, have you tried adding the `_schema_name: "
How can it helps me? How can it helps standard DatabasScheduler to go to tenants and find model with PeriodicTasks there?
Your variant may be helpfull if PeriodicTasks with tasks for all tenants placed in public
and we get schema for make context from kwargs. But this is not our variant of use
The DatabaseScheduler
pulls tasks from the public schema and sends them onto the queue, with given arguments. You can save the tasks schedule into the public schema with a proper _schema_name
kwarg so that the scheduler will send the celery task with a proper schema inside.
My customized DatabaseScheduler pulls tasks from all schemas, not only from public and send them onto queue. No need to save tasks from one schema to another, or make any other job. It just works. Your variant is simple and great. Customized DatabaseScheduler is just a tenant-aware replacement for standard DatabaseScheduler. In other cases your TenantAwareScheduler is very helpfull.
@Guest007 thank you for this code, could you please share your TENANT_APPS
and SHARED_APPS
settings? If i want to run the tasks in all tenants except public tenant how can I do that? because when I am trying to start celery beat it gives me migrations error (probabbly because I placed celery beat in TENANT_APPS
only.
@Guest007 thank you for this code, could you please share your
TENANT_APPS
andSHARED_APPS
settings?
For example in settings.py
PROJECT_APPS = [
"some_your_app",
...
]
INSTALLED_APPS = [
"django.contrib.admin",
...
"django_celery_beat",
....
] + PROJECT_APPS
TENANT_SPECIFIC_APPS = [
"django_tenants",
"customers_as_in_django_tenants_docs",
]
SHARED_APPS = TENANT_SPECIFIC_APPS + INSTALLED_APPS # in public schema
TENANT_APPS = INSTALLED_APPS # in tenant schemas for each client.
INSTALLED_APPS = SHARED_APPS
TENANT_MODEL = "customers_as_in_django_tenants_docs.Client"
TENANT_DOMAIN_MODEL = "customers_as_in_django_tenants_docs.Domain"
CELERY_BEAT_SCHEDULER = "celery_beat_tenants_scheduler.scheduler.TenantDatabaseScheduler"
If i want to run the tasks in all tenants except public tenant how can I do that? ...
You can put on top of task's body something like that:
from django.db import connection
def some_not_public_task():
if not hasattr(connection, "schema_name") or connection.schema_name == get_public_schema_name():
return
# body of all_tenants_task
Thanks again, what's the prupose of adding django_celery_beat
in both apps?
Thanks again, what's the prupose of adding
django_celery_beat
in both apps?
1) Using all apps in public schema (SHARED_APPS) is the easiest way to make tests 2) The 'public' schema is the same schema as all the others schemas/tenants. Therefore the app is included everywhere. You can do it differently
Hi! We make easy solution for tenants and schedules in database (like periodic tasks).
In case of:
you can't work in separate tenants with their own periodic tasks directly, without some
dispatcher task
or some separate model inpublic
tenant, likedjango-tenants-celery-beat
use.With customized DatabaseScheduler you can use celery-beat individually for each tenant as it was usual in non-tenant environment. Just remember that
django_celery_beat
must be in yourTENANT_APPS
.