support celery_queue_size metric for sentinels

[roganartu]

Implements #220.

This turned out to be much easier than I expected. Celery seems to just handle all the details of the Sentinel connection for us automatically.

Tested with a basic docker-compose stack running sentinel, where I killed the workers and then queued tasks. All the other metrics seemed fine, but the queue size correctly tracked the number of tasks I added as expected:

❯ curl localhost:9808/metrics -s | rg celery_queue
# HELP celery_queue_length The number of message in broker queue.
# TYPE celery_queue_length gauge
celery_queue_length{queue_name="default"} 1.0

Let me know if there's some other testing or something I should do here, or if there's something I'm missing given how surprisingly easy this was.

[danihodovic]

Hi Tony.

Given that this doesn't break existing behavior I don't see a problem merging this without tests. If you'd like to prevent future amendments to the code from breaking sentinel support feel free to add tests. If you are interested in this we could add a sentinel compose setup and a Github action matrix for testing sentinel.

Released as docker.io/danihodovic/celery-exporter:0.7.3

Thanks :beer:

[roganartu]

Amazing, thanks for speedy merge and release!

I have a boilerplate sentinel compose setup that I'd be happy to share if you'd like. It's fairly straightforward, but I'm not too concerned about the risk of breakages here tbh.

[ClementGautier]

@roganartu Would you mind sharing the docker-compose file here? I'm trying to make it work but it seems the exporter don't retrieve an empty list of queues in that part of the code: https://github.com/danihodovic/celery-exporter/blob/a84bd01fc83a74e80a8a599ae4f4e4c7ca3095d4/src/exporter.py#L205 I get the following if I try to do it manually in a python cli the connection work though: