Closed FrancoisPoinsot closed 5 years ago
So i figured that first thing i would need to check if i can handle uncommitted transaction correctly, is to have some uncommitted transaction. So here is a simple producer that output some committed and some uncommitted messages: https://github.com/FrancoisPoinsot/simplest-uncommitted-msg
Using previous custom producer, here is a setup for testing the read_commited: https://github.com/FrancoisPoinsot/sarama/tree/read-committed/tools/uncomitted-messages
Everything is in the readme, but in short:
With this, i supose next step is to try to actually implement the read_committed.
I think I found a pretty concise description of how client should handle read_committed: https://github.com/apache/kafka/blob/38a7708a47361929df37ed0f03160bbac5e51e93/clients/src/main/resources/common/message/FetchRequest.json#L58
Even though transaction are not ready yet in sarama, it would still be super useful to be able to read committed messages only. Of course such message would typically be produced by java producer to begin with.
Before spending too much time on this, i would like to confirm a few things first.
1) Sarama does not support yet
read_committed
on consumer side. Is it correct? 2) Sarama does not support yet transactions on producer side. Is it correct? 3)read_committed
is not just about sending the correct flag to the broker on afetchRequest
. The client need to filter the results. Is it correct? 4) It is totally doable to supportread_committed
without using transactions on producer side. The magic only happen on consumer side. Is it correct? 5) Is there some on-going work regarding theread_committed
support?When looking for information about how to support
read_committed
i stumbled upon 2 docs that i will note here as reference:However, even after reading those, the work done by the client is still unclear to me. I would gladly get my hand on any link that would describe accurately how to implement
read_committed
.