Add poison queues support for RabbitMQ

marcominerva commented 1 month ago

Motivation and Context (Why the change? What's the scenario?)

See #408.

marcominerva commented 3 weeks ago

I have updated this PR using Quorum Queues and Dead Letter Queues.

There are a couple of issues:

After creating the Quorum Queue:

this._channel.QueueDeclare(
    queue: this._queueName,
    durable: true,
    exclusive: false,
    autoDelete: false,
    arguments: new Dictionary<string, object>
    {
        ["x-queue-type"] = "quorum",
        ["x-delivery-limit"] = this._config.MaxRetriesBeforePoisonQueue,
        ["x-dead-letter-exchange"] = poisonExchange
    });

The delivery limit cannot be changed, otherwise an exception will be thrown. So, we cannot modify the value of MaxRetriesBeforePoisonQueue. To make this value dynamic, we should use a policy, but it seems that policies aren't supported by the RabbitMQ .NET client (the only way to configure them is making an HTTP call to RabbitMQ API).

Messages are requeued instantly, because there isn't a native way to have a delayed retry.

marcominerva commented 2 weeks ago

Hi @dluc! Have you got the chance to think about the issues I mentioned? Do you have any idea about how to handle them?

dluc commented 2 weeks ago

Hi @dluc! Have you got the chance to think about the issues I mentioned? Do you have any idea about how to handle them?

not yet sorry, other projects keeping me busy. I don't know when I'll be able to look through this - I'm hoping to merge the streaming PRs first, to improve the end user experience

microsoft / kernel-memory

Add poison queues support for RabbitMQ #648

Motivation and Context (Why the change? What's the scenario?)