Miksus / rocketry

Modern scheduling library for Python
https://rocketry.readthedocs.io
MIT License
3.26k stars 105 forks source link

DOCS - Django #76

Open mascDriver opened 2 years ago

mascDriver commented 2 years ago

Is it corrected in the latest version (master branch)? Yes

Latest version of the docs: https://rocketry.readthedocs.io/en/latest/

Location in the documentation you wish to improve https://rocketry.readthedocs.io/en/stable/examples/index.html

Describe the problem I didn't find examples for use with django, is it possible to use with django?

Suggestion for the fix Explain usage with Django.

Miksus commented 2 years ago

I have integrated the previous version to Flask (should be even easier with current state of the library) and I have integrated current version of Rocketry to FastAPI (https://github.com/Miksus/rocketry-with-fastapi) but unfortunately I'm not familiar with Django. I can take a look at some point a bit how Django works and see how it could be integrated.

You probably want Rocketry to start on the main loop of Django as a separate thread or using async (if Django fully supports it). If you use multiprocessing with Django, you probably want to create pipes to modify the Rocketry's environment (if you want Django routes to cxommand Rocketry's session).

mascDriver commented 2 years ago

basically my doubt is how do i start django with rocketry app.start(), because in fast api you have main function and you can start rocketry app there, but in django you can't

carlosm27 commented 2 years ago

I haven't used Rocketry with Django, yet. But did you try starting Rocketry in init.py? Like:

// __init__.py

from rocketry import Rocketry

app = Rocketry()

__all__ = app.run()
mascDriver commented 2 years ago

didn't work, application is waiting for something, i tried run with runserver, uvicorn, daphne

mascDriver commented 2 years ago

I tested a lot but the problem is that I can't start the app as in fastapi, because I would have to be changing manage.py, which is where the if main call is and I believe it's not recommended to do that, if I do as in the comment of @carlosm27 the server starts listening only to rocketry

carlosm27 commented 2 years ago

Hey @mascDriver, I was searching and found that a Django app can be overridden to run code from startup. Like this:

//apps.py

from django.apps import AppConfig

class YourAppConfig(AppConfig):
     name = 'Your app'

     def ready():
        from rocketry import Rocketry
        Rocketry.run()

Settings.py

...

INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'yourapp.apps.YourAppConfig' 
]

Here is some reference:

mascDriver commented 2 years ago

hey @carlosm27, not working yet, happens the same problem, when i started the server, with runserver, uvicorn or daphne, rocketry is the unique executing, django server not is started

Tobi-De commented 1 year ago

Hey @mascDriver, I haven't tried rocketry yet but I guess the easiest way to run something like this with django would be to run it in an entirely separate process with a django management command for example. Try creating a django management command that looks something like this:

from django.core.management.base import BaseCommand
from rocketry import Rocketry
from rocketry.conds import daily

app = Rocketry()

@app.task(daily)
def do_daily():
    ...

class Command(BaseCommand):
    help = "Run the rocketry app"

    def handle(self, *args, **options):
          app.run() 

Then you could use something like supervisor or systemd to run the command as a separate process. This would be like deploying your django application twice with the exact same configuration, but instead of running your django application with gunicorn (or whatever you're using), you'll run the management command :

# for example
python manage.py run_rocketry # run_rocketry being the management command name 

To keep things simple, I suggest using only synchronous rocketry tasks, django's asynchronous support is not very good at the moment. I hope this helps.

Btw @Miksus rocketry really looks awesome, thanks for this great work.

paolodina commented 1 year ago

@Tobi-De custom command + supervisor fit my needs too, however I wonder how to pass args and options to the task. Could you help?

Tobi-De commented 1 year ago

@Tobi-De custom command + supervisor fit my needs too, however I wonder how to pass args and options to the task. Could you help?

@paolodina I'm not very familiar with Rocketry, but since the tasks are run by the scheduler, I'm not sure if you can pass arguments to tasks programmatically. I found a few pages of rocketry docs that explain how to pass parameters to functions, maybe that would help.

https://rocketry.readthedocs.io/en/stable/tutorial/intermediate.html?highlight=param#set-output-as-an-input https://rocketry.readthedocs.io/en/stable/handbooks/parameterization.html

Can you give more details about your use case? I am curious to know what you are trying to achieve that requires an argument for your task.

paolodina commented 1 year ago

Thanks for the links, I'll read them again.

About the use case, I have a regular pre-existing Django command for importing stuff that I can run manually. Now I need to schedule its execution and I'd like to use Rocketry for that.

This is what I'm trying to do:

from django.core.management.base import BaseCommand
from rocketry import Rocketry

app = Rocketry(config={'task_execution': "main"})

class TaskMyImport:
    def __init__(self, *args, **options):
        real_run = options.get('real_run')

        if real_run:
            pass # do things
        else:
            pass # do other things

@app.task("every 5 seconds")
def task_my_import():
    TaskMyImport()

class Command(BaseCommand):
    def add_arguments(self, parser):
        parser.add_argument('--real-run', action='store_true', help="Read real data")

    def handle(self, *args, **options):
        # here (?) need a way to "pass" args and options
        app.run()
Tobi-De commented 1 year ago

Thanks for the links, I'll read them again.

About the use case, I have a regular pre-existing Django command for importing stuff that I can run manually. Now I need to schedule its execution and I'd like to use Rocketry for that.

This is what I'm trying to do:

from django.core.management.base import BaseCommand
from rocketry import Rocketry

app = Rocketry(config={'task_execution': "main"})

class TaskMyImport:
    def __init__(self, *args, **options):
        real_run = options.get('real_run')

        if real_run:
            pass # do things
        else:
            pass # do other things

@app.task("every 5 seconds")
def task_my_import():
    TaskMyImport()

class Command(BaseCommand):
    def add_arguments(self, parser):
        parser.add_argument('--real-run', action='store_true', help="Read real data")

    def handle(self, *args, **options):
        # here (?) need a way to "pass" args and options
        app.run()

I think in the scheduler task you will have to set your argument in the task definition, something like this:

@app.task("every 5 seconds")
def task_my_import():
    TaskMyImport(real_run=True)