search

rebus-org / Rebus.RabbitMq

:bus: RabbitMQ transport for Rebus
https://mookid.dk/category/rebus
Other
62 stars 44 forks source link

Support for Batch sending #107

Closed TsengSR closed 3 months ago

TsengSR commented 1 year ago

I've been wondering whether or not the messages are sent as batch, when using a transaction and ended up here

https://github.com/rebus-org/Rebus.RabbitMq/blob/2320f64385dc4913143d79c2839568f6969329cb/Rebus.RabbitMq/RabbitMq/RabbitMqTransport.cs#L729-L735

From what I understand is, that even though the acknowledges are batched (model.ConfirmSelect()), the messages itself aren't batched, right?

In my research I at least found out that some transport may support that and RabbitMQ is one of them according to this PR https://github.com/rabbitmq/rabbitmq-dotnet-client/pull/368

I wonder if we could have that in RabbitMQ Transport too, preferably as opt-in feature. It's not purely for performance, its more that'd I'd prefer to have either all or none messages being accepted by RabbitMQ rather than having half of them submitted/confirmed and the other half failing when the connection drops or similar.

It doesn't necessary need a public API for batching, just adding it to the transport and using it if the feature been enabled when confirming the message scope

mookid8000 commented 1 year ago

Oh, I didn't know that the RabbitMQ driver had added a batching API.

It should be fairly straightforward to have Rebus use that when it's relevant to do so,.

PRs are most welcome 🤗 otherwise sit tight and wait – I will fix this some time in the future.

mookid8000 commented 3 months ago

With Rebus.RabbitMq 9.2.0 it is now possible to enable batching by doing something like this in the configuration:

services.AddRebus(
    configure => configure
        .Transport(t => t.UseRabbitMq(connectionString, queueName)
            .SetBatchSize(100))
);

which will make Rebus use the driver's built-in ability to batch commands when possible.

When enabled, Rebus will automatically batch all messages sent/published from inside Rebus handlers or from RebusTransactionScopes.

Please note that it might not always be beneficial to enable batching. E.g. a few tests run with a RabbitMQ running in a local Docker container revealed no benefit from enabling batching. This might be different when using a RabbitMQ that is hosted on a different server or with a slower network.