search

monix / monix-kafka

Monix integration with Kafka
Apache License 2.0
123 stars 38 forks source link

Kafka Streams API support #32

Open mihaisoloi opened 6 years ago

mihaisoloi commented 6 years ago

Adding support for the Streams API[1] offers some benefits, and is well matched for Monix's strengths which would allow interoperability with the KStream DSL[2]. This could be implemented as a Monix KafkaStreamSubject in monix-kafka, as the Streams API is used for both consuming and publishing to topics.

[1] https://docs.confluent.io/current/streams/introduction.html#the-kafka-streams-api-in-a-nutshell [2] https://docs.confluent.io/current/streams/javadocs/org/apache/kafka/streams/kstream/KStream.html

alexandru commented 6 years ago

👍 but only if somebody commits to developing it

ryanworsley commented 6 years ago

Any thoughts on how state stores would be handled within the API without leaking the Kafka Streams abstractions through? I'm keen to help as this is something I'm probably going to do professionally in the near future anyway.

Avasil commented 6 years ago

I'm currently focused on helping with Monix 3.0.0 release so unfortunately can't think about it too much right now but @leandrob13 has started working on Kafka Streams support in #35 so maybe you can work together on this. :)

I'll try to give my input as soon as I can

leandrob13 commented 6 years ago

@Avasil @ryanworsley any help is welcomed! I just started defining config and wrappers, it seems kind of straight forward after looking at KStream API but I don't know if Iterant should fit in some places.

Focused on some issues regarding TaskLocal in monix but I will be engaging in this shortly again. Waiting for some feedback and pointers for the first approach.