Closed mocenas closed 4 years ago
@mocenas do you have a setup that I could use to reproduce this?
@gytis. Yes you can use this repoducer: https://gitlab.cee.redhat.com/mocenas/webflux-kafka-bug-reproduce
In your first application you manually assign a partition to a consumer. Therefore that consumer should not be in a consumer group. Remove vertx.kafka.consumer.group.id=log
from the application.properties
in that application.
Consumers groups are used to automatically organise which consumer will receive messages from which topic partition.
If you want to keep both your consumers in a same group, use subscribe
instead of assign
.
I'll add a note about this in a readme.
@gytis Thanks a lot, I overlooked the consumer groups configuration entirely. I thought that if these applications have different topics, they should not interfere. Well I will study about the consumer groups.
Changing from assign to subscribe a giving a separate consumer group seems to solve the problem. Thanks.
Connecting 2 pods with Webflux Kafka client to Kafka broker seems to cause problems. Testing with 1 pod, all is OK. Testing with 2 pods (different app, different Kafka topic) causes one of the pods to have some error in the log (full log springboot-webflux-kafka-1-54g55.log):
Also this pod does occasionally miss some Kafka messages. In my test I've sent messages 1,2, it received only message 2.