search

IBMStreams / streamsx.kafka

Repository for integration with Apache Kafka
https://ibmstreams.github.io/streamsx.kafka/
Apache License 2.0
13 stars 9 forks source link

KafkaProducer: Consistent region reset can trigger addtional reset #196

Closed ghost closed 4 years ago

ghost commented 4 years ago

On consistent region reset, the KafkaProducer is forcibly closed. When transactions are used, an on-going transaction is aborted before.

Following exceptions due to this handling can be observed:

17 Dec 2019 ... onRecordProduceException:130]  - Producer record 170 could finally not be produced for topic partition 'mySinglePartitionCCTopic-0' with 1 producer generations: org.apache.kafka.common.KafkaException: Failing batch since transaction was aborted

or 

17 Dec 2019 ... onRecordProduceException:126]  - Producer record 19 could not be produced for topic partition 'mySinglePartitionCCTopic-0' with 1 producer generations: org.apache.kafka.common.KafkaException: Producer is closed forcefully.

These exceptions are handled like all Kafka related exceptions are handled when in consistent region: A consistent region reset is initiated.

That's why a CR reset may be followed by another CR reset.

Expected behaviour: Exceptions caused by reset handling must be suppressed.

ghost commented 4 years ago

This unwelcome behavior has been introduced with implementation of the Producer's status port in v2.2.0

ghost commented 4 years ago

resolved in release 3.0.1