John-Connolly / SwiftQ

Distributed Task Queue
MIT License
257 stars 9 forks source link

Monitor for dead consumers #2

Open John-Connolly opened 7 years ago

John-Connolly commented 7 years ago

Currently SwiftQ does not monitor the processing queue for dead tasks. If a consumer crashes it will only be requeued on restart. SwiftQ uses per consumer processing queues consequently it is possible for tasks to get abandoned if that consumer is never restarted.

Possible Solutions

  1. Keep track of every time a consumer pops a task and periodically monitor for timeouts.

  2. Have task specific timeouts using the Redis SETEX command.

Monitoring

Every consumer will have to monitor every processing queue.