danihodovic / celery-exporter

A Prometheus exporter for Celery metrics
MIT License
399 stars 87 forks source link

celery_queue_length is not getting values always zero #190

Closed ahelmy closed 1 year ago

ahelmy commented 1 year ago

Thank you for this project. We are using it in our company, however this metric not working celery_queue_length always have zero. We are using redis.

ahelmy commented 1 year ago

@danihodovic your help please :)

danihodovic commented 1 year ago

Hi Ali. The queue length is working at my job. You will have to provide more information if we are going to debug this together.

ahelmy commented 1 year ago

I am using docker version 0.5.3. Do you think if used last version would matter?

danihodovic commented 1 year ago

The queue length only works for RabbitMQ. Are you using RabbitMQ?

ahelmy commented 1 year ago

No using redis. No way to make it work? Or a way to get the value

ahelmy commented 1 year ago

If no way please check this PR updating documentation https://github.com/danihodovic/celery-exporter/pull/191

danihodovic commented 1 year ago

Thank you!

paxw-panevo commented 1 year ago

What is meant in this ticket that it's not supported? Especially since the README has been updated to say that the celery_queue_length only works with RabbitMQ?

Our celery broker is redis and we can see some queue length data in our grafana dashboard,

image

...using this query,

image

ahelmy commented 1 year ago

Can you share celery configuration in python project, for example are you override default name of queue ?

paxw-panevo commented 1 year ago

We have different queues including the default queue called "default"

ahelmy commented 1 year ago

and is the default one showed in the queue_length metric ?

paxw-panevo commented 1 year ago

The default queue is showing the value of 0, true. But we were expecting that since we don't use that queue as much (if at all) and very likely the tasks are all consumed once celery queue length data is retrieved. At least that's what we think is going on based on what tasks we run and when.

ahelmy commented 1 year ago

I figured out the problem and created PR. Its about priority and queue names.