FinalsClub / karmaworld

KarmaNotes.org v3.0
GNU Affero General Public License v3.0
7 stars 6 forks source link

combine celery beat and celery worker into one python call #415

Closed btbonval closed 9 years ago

btbonval commented 9 years ago

celery worker has a -B flag to run beat concurrently. If this works as --help would suggest, we would no longer need celerywrapper.sh.

btbonval commented 9 years ago

Yeah. -B definitely runs Beat.

Let's see if I can get all this into the Procfile: python manage.py celery worker -B -l info -Q $CELERY_QUEUE_NAME

btbonval commented 9 years ago

Alright here's the weird thing. foreman start only runs the web: directive on my system, but according to the tutorial example, it should run every thing in there. http://blog.daviddollar.org/2011/05/06/introducing-foreman.html

Foreman is supposed to take a foreman start [PROCESS] argument, but I can type anything in for [PROCESS] where nothing happens, exit 0, even if the process is some completely random name. Not sure what that's about.

Maybe "worker", "web", and "clock" are special.

btbonval commented 9 years ago

foreman check was helpful. It only lists web, unless I specify -f Procfile. So what file is it using by default?

(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ foreman check -f ./Procfile
valid procfile detected (poo, celery, worker)
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ cat Procfile
poo: newrelic-admin run-program gunicorn -b 0.0.0.0:$PORT karmaworld.wsgi
celery: python manage.py celery worker -B -l info -Q $CELERY_QUEUE_NAME
worker: echo test!
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ foreman check
valid procfile detected (web)
btbonval commented 9 years ago

It's defaulting to Procfile-development. Why?

(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ foreman check -f Procfile-development 
valid procfile detected (web, other)
(karmanotes-web)vagrant@vagrant-ubuntu-trusty-64:/vagrant$ cat Procfile-development 
web: python manage.py runserver "0.0.0.0:$PORT" --settings "$DJANGO_SETTINGS_MODULE"
other: echo ""
btbonval commented 9 years ago

it is defined .foreman, which was ignored by my grep. Alright. I don't see any reason to maintain a web-only version. Just do foreman start web if that's what is called for.

btbonval commented 9 years ago

Ah. Procfile is for Heroku to run newrelic and all that. Trying to run Procfile directly locally led to some import errors with the web worker (no problems with celery and beat). The environment apparently didn't get setup right for using the wsgi file?

I don't like the divergent nature of local versus Heroku testing in this case. Better to align this more closely rather than diverge files which both need to be maintained.

btbonval commented 9 years ago

The import error is specific to wysihtml5.widgets. This is a new thing. Maybe it is not coincidental. I might need to do something special to support this, which is a problem I would have experienced when pushing to beta. If true, more reason not to run divergent Procfiles, or else problems like this go missed.