search

tulios / kafkajs

A modern Apache Kafka client for node.js
https://kafka.js.org
MIT License
3.7k stars 523 forks source link

EOS Support #155

Closed ianwsperber closed 5 years ago

ianwsperber commented 5 years ago

Has any consideration been given to support for Exactly-Once-Semantics? As far as I know presently no Node client supports EOS features (idempotent producer, producer/consumer transactions). If EOS is not on the roadmap today one might at least establish the level of effort required to implement these features and how they would effect the existing API.

If there's some way I can help establish the LoE please let me know :)

tulios commented 5 years ago

Hi @ianwsperber, we are wrapping up the work on version 1.4.0, which introduce Kafka v0.11 support, the infrastructure for EOS is implemented in this version. After the release, idempotent producers are our next priority. I will create a project for idempotent producers with issues describing the work needed.

ianwsperber commented 5 years ago

Ok, glad to hear it's on the roadmap @tulios! Having a project & tickets would be super helpful for me, as I'm trying to get a grasp on when I might expect to have transactions available in Node. If you could update this thread when the project has been created I'd appreciate it! Thanks.

tulios commented 5 years ago

@ianwsperber the project just contains the new protocol implementations, but I'll add the remaining issues soon.

https://github.com/tulios/kafkajs/projects/3

Feel free to re-open this issue

ianwsperber commented 5 years ago

Awesome, thanks @tulios! The team I'm working with has a need for transactions on their project. Would you accept PRs to implement these protocols? We may have some bandwidth to help.

tulios commented 5 years ago

Of course :D Right now I'm reading about how the APIs should interact and how we should implement the producer, I'm trying to create the remaining issues, but your help would be much appreciated.