search

nesquena / backburner

Simple and reliable beanstalkd job queue for ruby
http://nesquena.github.com/backburner
MIT License
428 stars 68 forks source link

redundant beanstalkd #122

Closed sakoula closed 6 years ago

sakoula commented 8 years ago

I am trying to build a distributed system that has 2 beanstalkd instances. I want this for redundancy reasons. Each job is enqueued to only one beanstalkd instance (using round robin). When all beanstalkd instances are up all of them are being used, when one or more of them are down, it is not being used.

I have seen that beaneater removed this functionality of their code in the latest version. backburner seems to rely on beaneater in order to handle beanstalkd connectivity. Are you thinking of adding such functionality to your code or will you really on the beaneater implementation for this, or is their another way to implement this?

eltone commented 7 years ago

One way to do this would be to enqueue jobs via a load balancer (ELB or HAProxy etc). To process jobs you can either create a new worker that supports this or start one backburner process per beanstalkd and connect directly.