Closed KrzysztofMadejski closed 7 years ago
I've done 2 processes for celery, let me know if you have better idea.
; ===============================
; ckan celeryd supervisor example
; ===============================
; symlink or copy this file to /etc/supervisr/conf.d
; change the path/to/virtualenv below to the virtualenv ckan is in.
[program:celery-priority]
; Full Path to executable, should be path to virtural environment,
; Full path to config file too.
command=/home/ckan/.virtualenvs/ckan/bin/paster --plugin=ckanext-archiver celeryd2 run priority --config=/etc/ckan/prod.ini
; user that owns virtual environment.
user=ckan
numprocs=1
stdout_logfile=/var/log/ckan/celeryd.log
stderr_logfile=/var/log/ckan/celeryd.log
autostart=true
autorestart=true
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998
[program:celery-bulk]
; Full Path to executable, should be path to virtural environment,
; Full path to config file too.
command=/home/ckan/.virtualenvs/ckan/bin/paster --plugin=ckanext-archiver celeryd2 run bulk --config=/etc/ckan/prod.ini
; user that owns virtual environment.
user=ckan
numprocs=1
stdout_logfile=/var/log/ckan/celeryd.log
stderr_logfile=/var/log/ckan/celeryd.log
autostart=true
autorestart=true
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998
Yep that's what we did - run it twice with celeryd2.
Ok, I've added a suggested template with two definitions and updated the docs. All in #44 PR ready to be merged.
In
readme.md
those links are broken:I've checked following links:
but none of them specifies how to setup two queues with supervisor. @davidread do you have any working example of such supervisor config file?
BTW: Warning: in 2.7 a new background job system has been introduced. As such the archiver probably won't with new CKAN. See #42