search

bee-queue / arena

An interactive UI dashboard for Bee, Bull and BullMQ packages
MIT License
844 stars 235 forks source link

discovery of queue ? #127

Open maxired opened 5 years ago

maxired commented 5 years ago

Hey,

from what I understand so far, it looks it need to configure all the queue that I want to monitor. It that right ?

The application I am working on is creating queues on the fly, so I was wondering whether I could dynamically monitor all queues, starting with specific prefix ?

ttacon commented 5 years ago

Could you expand a bit more on your use case here? Definitely interested! We use an auto-discovery method ourselves internally at the moment, but it requires auto-restart of arena. Would love to make it more reactive!

dwightkelly commented 5 years ago

We create queues on demand for specific complex jobs. This means we can't add them all to config index.json file.

maxired commented 5 years ago

Hey @ttacon sorry, been off for a while ;)

I use bee queue for an ecommerce project I am working on. Basically, computing the price is the job done in the queue.

What we do is creating a new queue for each different orders. This allow us to control were the job is processed and so on

JonDum commented 5 years ago

Another use case here. Each "queue" is a different pub/sub message type in one of our services and there are many of them. Would be great if arena could just figure out all the queues without having to specify them in the json.

shortcircuit3 commented 5 years ago

Same here! We generate queues for each type of job and new job types get added all the time

adamsoffer commented 5 years ago

Same here. I create queues on the fly dynamically so I can't account for them all when running Arena. @ttacon is it possible to show queues in arena as they are created?

DanielChuDC commented 4 years ago

same here. Required to create queue dynamically. Any idea?

kevzettler commented 4 years ago

even without having dynamic queue creation its just a pain in the ass to configure and maintain the index.json. If developers add or remove queues to our application we need to manually update the index, redeploy and restart arena.

there is this post from Mixmax: https://engineering.mixmax.com/blog/bull-queue-aws-autodiscovery

in which they too feel this pain and have created a helper script to pull a list of queues from redis on aws.

Why is this functionality is not built into Arena?