IBMStreams / streamsx.kafka

Repository for integration with Apache Kafka
https://ibmstreams.github.io/streamsx.kafka/
Apache License 2.0
13 stars 9 forks source link

Optimize consistent region reset with only one consumer in consumer group #186

Closed ghost closed 4 years ago

ghost commented 5 years ago

When the consumer group has only one consumer, the reset implementation can be simplified. Multiple operator checkpoints need not be merged into one group checkpoint via the group MX bean.

How to detect, that only one consumer needs to be reset in a consumer group:

Every consumer operator is registers in the group MX bean in setup with its operator name when the connection to the JCP is created. https://github.com/IBMStreams/streamsx.kafka/blob/bd2d899e554fcef4260565c04973cbd4aada1769/com.ibm.streamsx.kafka/impl/java/src/com/ibm/streamsx/kafka/clients/consumer/CrKafkaConsumerGroupClient.java#L232 The registered operators are part of every operator's checkpoint. When the checkpoint contains only the own operator name, the optimization applies.

ghost commented 4 years ago

resolved in Release 3.0.0.