New Feature Request: Add optional back-off option to consumer

[EndangeredAcorn]

Problem

For small queues if a required service goes down for the consumer to function properly and causes errors or a temporary issue with a particular message, the message gets put back into the queue and almost immediately re-consumed hence making a cycle.

Proposed Solution

Optional back-off option to the consumer constructor which adds a non-blocking wait to the consumer before requeuing the message.

const consumer = Rabbit.createConsumer(
  {
    queue: "tasks",
    queueOptions: {
      ...
    },
    exchanges: [...],
    backOff: 60 // When ConsumerStatus.REQUEUE, wait 60 seconds before actually requeuing the message
                      // meanwhile continue processing next message
    ...
  },

[cody-greene]

I think the correct behavior is to return the message ASAP so healthy consumers (if any) can take over the work. So if I were to implement this, I'd have to cancel the consumer and either nack all prefetched messages or just hand them off to the handler function normally and let them fail so they get NACK'd anyway. I would have to add a few more configuration options though because this is a kind of rate limiting: backoff strategy (exponential backoff or linear), backoff conditions (different ConsumerStatus), failLimitInterval and failLimitBurst (trigger when jobs which fail more than X times within an interval time span). This is looking like a whole can of worms.

I could also just add consumer.start() as a counterpart to consumer.close() and leave it up to users to decide when to temporarily stop consuming. They'd have to clean up their own timers but this is far more flexible and allows precise control.

To be clear, I'm not opposed to pausing the consumer automatically. I'd just have to spend some time to get it right.

[EndangeredAcorn]

I could also just add consumer.start() as a counterpart to consumer.close() and leave it up to users to decide when to temporarily stop consuming. They'd have to clean up their own timers but this is far more flexible and allows precise control.

That sounds like a sensible approach. Adding a consumer.start() method as a counterpart to consumer.close() would provide a flexible solution for handling scenarios where a service that the consumer depends on goes down.

Overall I believe this would a great solution to prevent the problems of adding the backoff function and would solve most of the problems related to it.

Feel free to close the issue

Have a nice day ahead!

[cody-greene]

Consumers can be restarted as of v4.6.0 with consumer.start()