micronaut-projects / micronaut-kafka

Integration between Micronaut and Apache Kafka
Apache License 2.0
86 stars 107 forks source link

Documentation update for multiple kafka streams #1022

Open dradosevic opened 7 months ago

dradosevic commented 7 months ago

Issue description

Micronaut Kafka documentation for using multiple kafka streams (https://micronaut-projects.github.io/micronaut-kafka/latest/guide/#kafkaStreams ; 'Configuring multiple Stream definitions on the same Micronaut Service.' section) does mention that It is advisable when using multiple streams in a single app to provide a @Named instance of ConfiguredStreamBuilder for each stream. but the code examples @Singleton @Named("my-other-stream") KStream<String, String> myOtherKStream(ConfiguredStreamBuilder builder) { return builder.stream("my-other-stream"); }

and

@Singleton @Named("my-stream") KStream<String, String> myStream(ConfiguredStreamBuilder builder) {

do not follow that recommendation. When app with multiple streams without this annotation is started, there is an error: 13:41:45.261 [main] ERROR o.a.k.s.p.internals.TopologyMetadata - stream-client [tdrstorage-7ea31531-fbca-4539-b15c-b941a9298f1d] Topology with no input topics will create no stream threads and no global thread.

This can be reproduced with https://github.com/dradosevic/kafka-streams-example project. It contains an example of multi-stream app which starts successfully, but if you remove Named annotation from ConfiguredStreamBuilder parameter and place it at the method level, this error will occur during startup.

Also, application.id had to be added to the config to successfully start the app (a good explanation of these issues is at https://github.com/micronaut-projects/micronaut-kafka/issues/139#issuecomment-688956724), maybe that can be included in the documentation as well.

sdelamo commented 7 months ago

@dradosevic are you willing to provide a PR to improve the documentation?