ryanhiebert / hirefire

A Python lib to integrate with the HireFire service -- The Heroku Proccess Manager.
https://hirefire.readthedocs.io/
Other
34 stars 21 forks source link

Problem running test with Celery #1

Closed jacksonofalltrades closed 9 years ago

jacksonofalltrades commented 11 years ago

When I try this: http://localhost:8000/hirefire/<HIREFIRE_TOKEN>/info

I get the following exception: Traceback (most recent call last): File "/Library/Python/2.7/site-packages/django/core/handlers/base.py", line 89, in get_response response = middleware_method(request) File "/Library/Python/2.7/site-packages/hirefire/contrib/django/middleware.py", line 64, in process_request return self.info(request) File "/Library/Python/2.7/site-packages/hirefire/contrib/django/middleware.py", line 52, in info 'quantity': proc.quantity() or 'null', File "/Library/Python/2.7/site-packages/hirefire/procs/celery.py", line 68, in quantity return sum([self.channel._size(queue) for queue in self.queues]) AttributeError: 'Channel' object has no attribute '_size'

Any ideas? I haven't found any reference to this problem on the web. Thanks.

jacksonofalltrades commented 11 years ago

Oh, I should mention, the middleware seemed to work fine (when I ran the other localhost in your docs - http://localhost:8000/hirefire/test)

jezdez commented 11 years ago

Hmm, interesting, can you paste your celery config, e.g. which backend do you use? Also, which version of Celery is that?

jacksonofalltrades commented 11 years ago

I've already moved off of Heroku on to AWS directly. I was using RabbitMQ backend with django-celery.

On Fri, Mar 22, 2013 at 2:24 AM, Jannis Leidel notifications@github.comwrote:

Hmm, interesting, can you paste your celery config, e.g. which backend do you use? Also, which version of Celery is that?

— Reply to this email directly or view it on GitHubhttps://github.com/jezdez/hirefire/issues/1#issuecomment-15287521 .

jezdez commented 11 years ago

Oh, okay, thanks @jacksonofalltrades. Can you remember which version of django-celery? The latest?

jacksonofalltrades commented 11 years ago

django-celery 3.0.11

On Sun, Mar 24, 2013 at 12:51 PM, Jannis Leidel notifications@github.comwrote:

Oh, okay, thanks @jacksonofalltradeshttps://github.com/jacksonofalltrades. Can you remember which version of django-celery? The latest?

— Reply to this email directly or view it on GitHubhttps://github.com/jezdez/hirefire/issues/1#issuecomment-15367048 .

ScottEAdams commented 10 years ago

I seem to be running into the same issue. Im running latest versions of everything.

ERROR Internal Server Error: /hirefire/XXXXXXXXX/info
Traceback (most recent call last):
  File "/workspace/project/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 90, in get_response
    response = middleware_method(request)
  File "/workspace/project/venv/local/lib/python2.7/site-packages/hirefire/contrib/django/middleware.py", line 64, in process_request
    return self.info(request)
  File "/workspace/project/venv/local/lib/python2.7/site-packages/hirefire/contrib/django/middleware.py", line 52, in info
    'quantity': proc.quantity() or 'null',
  File "/workspace/project/venv/local/lib/python2.7/site-packages/hirefire/procs/celery.py", line 67, in quantity
    return sum([self.channel._size(queue) for queue in self.queues])
AttributeError: 'Channel' object has no attribute '_size'

Works fine on the test url - http://localhost:8000/hirefire/test

ScottEAdams commented 10 years ago

My celery.py looks like this

from celery import Celery
from django.conf import settings
from hirefire.procs.celery import CeleryProc

app = Celery('project')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(settings.INSTALLED_APPS, related_name='tasks')

class WorkerProc(CeleryProc):
    name = 'worker'
    queues = ['worker']
    app = app

And celery is being started like this

python manage.py celery -E --app=project worker -l info
ScottEAdams commented 10 years ago

Seems that the librabbitmq channel object doesnt report _size

https://github.com/celery/librabbitmq/blob/master/librabbitmq/__init__.py

Switched to ironmq to test and it works

https://github.com/iron-io/iron_celery/blob/master/iron_celery/iron_mq_transport.py

mulka commented 10 years ago

If you are using RabbitMQ, you might be able to get the queue length through the management API using pyrabbit.

import pyrabbit c = pyrabbit.api.Client('host:port', 'username', 'password') c.get_queue_depth('vhost', 'celery')

mjtamlyn commented 9 years ago

This would be fixed by #9

mjtamlyn commented 9 years ago

Resolved by 9dc36d4b969adf92803729848f8118a1b4a6103a