Koed00 / django-q

A multiprocessing distributed task queue for Django
https://django-q.readthedocs.org
MIT License
1.84k stars 289 forks source link

how to integrate django-q with django as a software package #703

Open flyly0755 opened 2 years ago

flyly0755 commented 2 years ago

We have a destop application, which backend uses django. Every time provide a new version will package a software pack. Now I want to introduce django-q to our project for async operation. Usually python manage.py qcluster with this command to raise django-q worker. but with software pack, can't find standalone python executable file(for example python3.exe in windows) coz it is packed in some other executable file: such as a file called manage. when to run manage qcluster, it raise error Unknown command: 'qcluster'. Did you mean sqlclear? which i think interpreted as python manage.py sqlclear. with development mode, instead of software pack, run python manage.py qcluster is successfully, without problem.

openHBP commented 2 years ago

on Linux, you can use systemd. More info https://stackoverflow.com/questions/65241859/using-django-q-in-production

flyly0755 commented 2 years ago

on Linux, you can use systemd. More info https://stackoverflow.com/questions/65241859/using-django-q-in-production

which i think is unworkable, what i want is after runserver to execute python manage.py qluster, where i can add this extra operation? manage.py?

pysean3 commented 2 years ago

Not sure exactly what you're trying to do, and strongly recommend against this, but you can run those two commands at the same time in bash with an ampersand.

python manage.py runserver & python manage.py qcluster

flyly0755 commented 2 years ago

yes, can run with command python manage.py runserver & python manage.py qcluster, but now has another question, is there any command used for solely closing django-q process, for example python manage.py djangoq-close, then will close all the processes related with django-q? coz now django-q and django are binded in one software package, when close the django service, also need close the django-q service.

openHBP commented 2 years ago

You can serve your django application with Gunicorn also served by systemd.

You just need to setup 2 services file in /etc/systemd/system: 1 to run the server as explained here and 1 to run django-q cluster as explained here

runserver service

  1. install gunicorn in your virtualenv
  2. got to /etc/systemd/system and create a service file named myproject.service
  3. run systemctl start myproject.service

qcluster service

[Unit]
Description=myproject qcluster daemon
After=network.target

[Service]
User=myproject
Group=www-data
WorkingDirectory=/opt/pipenv/myproject/source
ExecStart=/home/myproject/.venv/bin/python manage.py qcluster
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
PrivateTmp=true
#StandardOutput=file:/var/log/myproject/qcluster.std.log
StandardError=file:/var/log/myproject/qcluster.log

[Install]
WantedBy=multi-user.target

StandardOutput seems not working...

systemd

Yo can start, stop, restart, enable, disable your services as you want. More info on systemd: https://www.linode.com/docs/guides/what-is-systemd/

And do not forget to run systemctl daemon-reload whenever your change anything in one of you 2 services files

flyly0755 commented 2 years ago

You can serve your django application with Gunicorn also served by systemd.

You just need to setup 2 services file in /etc/systemd/system: 1 to run the server as explained here and 1 to run django-q cluster as explained here

runserver service

  1. install gunicorn in your virtualenv
  2. got to /etc/systemd/system and create a service file named myproject.service
  3. run systemctl start myproject.service

qcluster service

[Unit]
Description=myproject qcluster daemon
After=network.target

[Service]
User=myproject
Group=www-data
WorkingDirectory=/opt/pipenv/myproject/source
ExecStart=/home/myproject/.venv/bin/python manage.py qcluster
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
PrivateTmp=true
#StandardOutput=file:/var/log/myproject/qcluster.std.log
StandardError=file:/var/log/myproject/qcluster.log

[Install]
WantedBy=multi-user.target

StandardOutput seems not working...

systemd

  • capable of starting services in parallel
  • capable of handling hot-plug devices; such as starting up services on demand at the occurrence of an event (like upstart)

Yo can start, stop, restart, enable, disable your services as you want. More info on systemd: https://www.linode.com/docs/guides/what-is-systemd/

And do not forget to run systemctl daemon-reload whenever your change anything in one of you 2 services files

Seems not ok, what i want is when i close the destop software(for example click x in the right-top), then all the django-q related processes to be killed.