Open AlexeyRaga opened 3 weeks ago
Is it possible that the problem can be in how middlewares are handled?
I see that BatchProduceAsync executes Produce
synchronously for each message, separately.
But it internally it does _middlewareExecutor.Execute
which returns Task
. This Task
is dropped on the floor.
This could mean that multiple Task
s are just running in parallel, calling the "real" producer deep inside, and by chance can finish out-of-order.
If I am right, then this execution model is unsound.
That is exactly why is not guaranteeing in order producing. For it to guarantee in order producing, the code would need to wait until InternalProduce is called (this method pushes the message to the driver buffer).
It should be very easy to reproduce this issue by adding a producing middleware with a descending timeout and see an inverse order of producing.
Let's say 3 messages with descending timeouts of 1s, 500ms, 0s.
Prerequisites
Description
It looks like
BatchProduceAsync
can produce message out-of-order. It happens from time to time, and the order doesn't seem to be deterministic. Most of the time it does seem to produce messages in the right order, but not always.Steps to reproduce
I was using this code to reproduce:
I then look at the topic itself and see that some batches come out of order.
Expected behavior
I expect that the messages order is guaranteed by the batch producer all the time.
Actual behavior
Most often messages are coming in the right order:
1, 2, 3
but sometimes they are written in the wrong order.KafkaFlow version
v3.0.10