Koed00 / django-q

A multiprocessing distributed task queue for Django
https://django-q.readthedocs.org
MIT License
1.83k stars 284 forks source link

Would it be useful to provide some decorator syntax? #323

Open edmenendez opened 5 years ago

edmenendez commented 5 years ago

Maybe something like this? It would allow you to call slow_function() and also slow_function.delay(). Using Celery like syntax without the Celery bloat.

import functools
from django_q.tasks import async_task

class Delay(object):
    """ Use this to wrap a function for Django Q. You can't use regular @
    decorator syntax. Instead you need to do: my_func = Delay(my_func_work).
    """

    def __init__(self, original_function):
        self.original_function = original_function
        functools.update_wrapper(self, original_function)

    def __call__(self, *args, **kwargs):
        return self.original_function(*args, **kwargs)

    def delay(self, *args, **kwargs):
        return async_task(self.original_function, *args, **kwargs)

def slow_function_work(arg1='world'):
    """ This is an example function.
    """
    return {'hello': arg1}

slow_function = Delay(slow_function_work)

Great work with django-q btw!

edelvalle commented 5 years ago

I have a function that I can use like this:

delayed(slow_function_work)(param1, param2)

or

Schedule the execution in 20 seconds:

delayed(slow_function_work, eta=20)(param1, param2)

Schedule the execution in 2 days form now:

delayed(slow_function_work, eta=timedelta(days=2))(param1, param2)

Schedule the execution at a specific date

delayed(slow_function_work, eta=datetime(year=2018, month=1, day=1, hour=2, minute=2))(param1, param2)

And it submits the task always after commit... so the task does not start before the current transaction closes. Because could be that you need model instances that are not yet saved and the tasks starts early and then bam!

Here is my implementation:


from datetime import timedelta
from functools import partial

from django.conf import settings
from django.db.transaction import on_commit
from django.utils.timezone import now
from xoutil.names import nameof

def delayed(task, eta=None):
    if getattr(settings, 'Q_CLUSTER_EAGER', False):
        return task
    else:
        def delayed(*args, **kwargs):
            from django_q.tasks import async_task, schedule

            set_eta = eta

            if isinstance(set_eta, int):
                set_eta = timedelta(seconds=set_eta)

            if set_eta and isinstance(set_eta, timedelta):
                set_eta = now() + set_eta

            if set_eta:
                f = partial(
                    schedule,
                    func=nameof(task, inner=True, full=True),
                    next_run=set_eta,
                    *args, **kwargs
                )
            else:
                f = partial(
                    async_task,
                    task,
                    *args, **kwargs
                )
            on_commit(f)

        return delayed
Koed00 commented 5 years ago

Great ideas. I'll look into this. Would be a nice addition. I've actually had a need for this a few times but then ended up writing my own code for it.

1oglop1 commented 3 years ago

Great ideas. I'll look into this. Would be a nice addition. I've actually had a need for this a few times but then ended up writing my own code for it.

@Koed00 Could you share the example, please? We are building an application with a lot of debt and code moves around rather quickly as we go through the refactoring.

Specifying func argument as plain string does not cause an immediate error.