search

nodefluent / kafka-streams

equivalent to kafka-streams :octopus: for nodejs :sparkles::turtle::rocket::sparkles:
https://nodefluent.github.io/kafka-streams/
MIT License
826 stars 111 forks source link

When using multiple input topics, does different message rate on topics affect streams processing? #143

Open priyag86 opened 4 years ago

priyag86 commented 4 years ago

I have 10 input topics (a topic per mysql table) that I am reading from in my kafka streams app, certain topics have very low message rate while other have slightly higher message rate. Wonder if kafka streams processing on faster topics will be stalled due to low message rate on slower topics?

krystianity commented 4 years ago

No they should not, the brokers will manage the different loads for you automatically, that is why I also suggest to consume multiple topics with the same consumer instance, instead of spawning multiple instances for each topic.