alexreidy / parallafka-for-dotnet

Parallelized Kafka consumer for maximizing throughput
Apache License 2.0
1 stars 1 forks source link

Parallafka order guarantee: partition or key #9

Open Gonyoda opened 2 years ago

Gonyoda commented 2 years ago

Parallafka currently only supports key order guarantee (all messages for the same key are received in order). Normal kafka supports partition order guarantee (all messages in a partition are received in order).

I believe a small change would be needed for Parallafka to support partition order guarantee. Granted, the parallelism will be limited to the number of partitions and overall message throughput will suffer. But supporting partition order will allow consumers to be written where the partition order is more important than the key order.