search

akka / alpakka-kafka

Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
https://doc.akka.io/docs/alpakka-kafka/current/home.html
Other
1.42k stars 387 forks source link

Document adding a buffer before transactional producer #1282

Open seglo opened 3 years ago

seglo commented 3 years ago

Short description

Document adding a buffering stage before the transactional producer so that some amount of messages can still be processed by user business logic while a commit is occurring in the producer stage (and back-pressuring).

This may help to improve throughput of transactional workflows in some cases.

jrudolph commented 3 years ago

Maybe it would be enough to document the implications of the transactional behavior on the user stream, instead? And add a snippet that people can experiment with a buffer themselves?

seglo commented 3 years ago

Good idea. We have enough of a maintenance burden with transactions as it is.

ennru commented 3 years ago

Let's try it in the benchmarks. (not that they'd have a lot of business logic, ofc)