Unknown command celeryd for biocloudcentral/manage.py

[MatthewRalston]

Hello again, I took advice from #42, setting up a fresh instance on a linux workstation, RHEL6 with pyenv-virtualenv. I took your suggestion @afgane and decided to install according to the repository's instructions. I manage fine until I receive the following error while starting supervisor. I notice that the celeryd command line tool is available(see below), although this subcommand is not available to manage.py

From the supervisor log (/srv/cloudlaunch/cloudlaunch/supervisor.log):

14:01:51 system   | web.1 started (pid=1543)
14:01:51 system   | celery.1 started (pid=1544)
14:01:52 celery.1 | Unknown command: 'celeryd'
14:01:52 celery.1 | Type 'manage.py help' for usage.
14:01:52 system   | celery.1 stopped (rc=1)
14:01:52 system   | sending SIGTERM to web.1 (pid 1543)
14:01:52 web.1    | Traceback (most recent call last):
14:01:52 web.1    |   File "../.cl/bin/gunicorn", line 11, in <module>
14:01:52 web.1    |     sys.exit(run())
14:01:52 web.1    |   File "/home/ralstonm/.pyenv/versions/cl/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
14:01:52 web.1    |     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
14:01:52 web.1    |   File "/home/ralstonm/.pyenv/versions/cl/lib/python2.7/site-packages/gunicorn/app/base.py", line 189, in run
14:01:52 web.1    |     super(Application, self).run()
14:01:52 web.1    |   File "/home/ralstonm/.pyenv/versions/cl/lib/python2.7/site-packages/gunicorn/app/base.py", line 72, in run
14:01:52 web.1    |     Arbiter(self).run()
14:01:52 web.1    |   File "/home/ralstonm/.pyenv/versions/cl/lib/python2.7/site-packages/gunicorn/arbiter.py", line 197, in run
14:01:52 web.1    |     self.halt()
14:01:52 web.1    |   File "/home/ralstonm/.pyenv/versions/cl/lib/python2.7/site-packages/gunicorn/arbiter.py", line 297, in halt
14:01:52 web.1    |     self.stop()
14:01:52 web.1    | 
14:01:52 system   | web.1 stopped (rc=1)

/srv/cloudlaunch >cloudlaunch/biocloudcentral/manage.py --help
Usage: manage.py subcommand [options] [args]

Options:
  -v VERBOSITY, --verbosity=VERBOSITY
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=all output
  --settings=SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath=PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Print traceback on exception
  --version             show program's version number and exit
  -h, --help            show this help message and exit

Type 'manage.py help <subcommand>' for help on a specific subcommand.

Available subcommands:
  cleanup
  compilemessages
  createcachetable
  dbshell
  diffsettings
  dumpdata
  flush
  inspectdb
  loaddata
  makemessages
  reset
  runfcgi
  runserver
  shell
  sql
  sqlall
  sqlclear
  sqlcustom
  sqlflush
  sqlindexes
  sqlinitialdata
  sqlreset
  sqlsequencereset
  startapp
  startproject
  syncdb
  test
  testserver
  validate

/src/cloudlaunch >cd cloudlaunch

/srv/cloudlaunch/cloudlaunch >/bin/ls ../.cl/bin
activate      bundle_image  celeryd        cwutil        dynamodb_load      fetch_file   gunicorn_paster  list_instances  pidproxy    python     sdbadmin
activate.csh      camqadm   celeryd-multi  django-admin.py   easy_install       get-pip.py   honcho       lss3        pip         python2    supervisorctl
activate.fish     celery    celeryev       django-admin.pyc  easy_install-2.7   glacier      instance_events  mturk       pip2        python2.7  supervisord
activate_this.py  celerybeat    cfadmin        djcelerymon   echo_supervisord_conf  gunicorn     kill_instance    nosetests   pip2.7      route53    taskadmin
asadmin       celeryctl cq         dynamodb_dump     elbadmin       gunicorn_django  launch_instance  nosetests-2.7   pyami_sendmail  s3put      wheel

[afgane]

This could have something to do with RHEL as I've only installed this on Ubuntu but nothing jumps at me.

With the virtualenv activated, what do you see when you issue pip freeze command? If celery is missing, try installing it via pip install celery. This is what I have:

(.cl)launch@athena:/srv/cloudlaunch/cloudlaunch$ pip freeze
Django==1.3
Jinja2==2.7
MarkupSafe==0.18
PyYAML==3.11
South==0.7.3
amqp==1.0.12
anyjson==0.3.3
argparse==1.2.1
billiard==2.7.3.31
bioblend==0.6.2-dev
boto==2.38.0
celery==3.0.21
dj-database-url==0.2.1
django-bootstrap-toolkit==2.12.0
django-celery==3.0.17
gunicorn==17.5
honcho==0.6.6
kombu==2.5.12
mock==1.0.1
nose==1.3.0
poster==0.8.1
psycopg2==2.4.2
python-dateutil==2.1
python-memcached==1.48
pytz==2013b
requests==2.7.0
requests-toolbelt==0.4.0
simplejson==3.3.0
six==1.9.0
wsgiref==0.1.2

While you're at it, I'd sugges updating bioblend to what's in git as some of the recent changes to Cloud Launch require the still unreleased bioblend changes. Just use this command: pip install -U git+https://github.com/galaxyproject/bioblend.git

[MatthewRalston]

I installed the updated bioblend changes as requested with no luck. I'm happy to supply whatever else. RHEL is frustrating indeed. I'm not sure why the output of manage.py --help is such... do you see something else in your build?

/srv/cloudlaunch >pip freeze amqp==1.0.12 anyjson==0.3.3 billiard==2.7.3.31 bioblend==0.6.1 boto==2.38.0 celery==3.0.21 dj-database-url==0.2.1 Django==1.3 django-bootstrap-toolkit==2.12.0 django-celery==3.0.17 gunicorn==19.3.0 honcho==0.6.6 Jinja2==2.7 kombu==2.5.12 MarkupSafe==0.18 meld3==1.0.2 mock==1.0.1 nose==1.3.0 poster==0.8.1 psycopg2==2.4.2 python-dateutil==2.1 python-memcached==1.48 pytz==2013b0 PyYAML==3.10 requests==1.2.3 requests-toolbelt==0.4.0 simplejson==3.3.0 six==1.9.0 South==0.7.3 supervisor==3.1.3 wheel==0.24.0

[MatthewRalston]

@afgane is does your manage.py script have the "celeryd" command?

[afgane]

Yes it does.

I'm working on an Ansible role to automate the installation process and hopefully help that way. In a basic format, it should be ready today but at least in this iteration it'll still be Ubuntu-only. Can you get a Ubuntu-VM instead of the RHEL?

[afgane]

Here's the Ansible playbook I mentioned: https://github.com/afgane/ansible-cloudlaunch

If you can get access to a Ubuntu machine, feel free to try it out and let me know if you run into any problems. If you'd like to add support for installing the app on a CentOS machine via a PR, we'd appreciate it (or you can send me required system package names and I'll eventually add it).

[afgane]

I also notice that you have some extra python libs and also that some are different versions than what's provided in requirements.txt. Did you install this in a virtualenv?

[MatthewRalston]

It turns out something went wrong with my virtualenv that was triggering the issue. I recreated the virtualenv, got supervisor working, and nginx launched. I encountered another issue with the celery.d process terminating with no error to the supervisor log. Thanks for helping me fix this issue @afgane