search

IBMStreams / streamsx.kafka

Repository for integration with Apache Kafka
https://ibmstreams.github.io/streamsx.kafka/
Apache License 2.0
13 stars 9 forks source link

KafkaConsumer: Function of the Control port, when used in Consistent Region? #184

Closed ghost closed 4 years ago

ghost commented 4 years ago

The function of the control input port is not documented when the operator is part of a consistent region. It is unclear, where the consistent region can begin. It is also unclear, if this scenario is supported at all.

ghost commented 4 years ago

Current implementation (toolkit version 2.2.1 and older)

The consumer operator must be the trigger operator of the consistent region (operator driven or periodic).

The trigger operator cannot be upstream of the control port:

CDISP0834E ERROR: Illegal consistent region topology: operator ReceivedMessages has exclusively control input ports and has output ports.

CDISP0092E ERROR: Because of previous compilation errors, the compile process cannot continue.

This means that an assignment or unassignment fo topic partitions done via control tuple can get reverted by resetting the consistent region.

Suggestion: Remove support of consistent region when a control input port is used.