I am using amazon-kclpy==2.0.6 to process kinesis streams. This works well for single stream processing. From the Java library, I see that multi-stream support was introduced in the java library version 2.3.0. Also worth noting that amazon-kclpy v2.0.6 uses Amazon Kinesis Client v2.3.9 as dependency.
The official docs released with the Java library suggests
With this new capability, you can update the list of streams at runtime for multi-stream processing in a scalable KCL application without redeploying the application
My use case requires one application to listen/process multiple Kinesis streams. As the stream names are static, could even provide the stream names as part of the configuration itself.
I am using
amazon-kclpy==2.0.6
to process kinesis streams. This works well for single stream processing. From the Java library, I see that multi-stream support was introduced in the java library version 2.3.0. Also worth noting thatamazon-kclpy v2.0.6
uses Amazon Kinesis Client v2.3.9 as dependency.The official docs released with the Java library suggests
My use case requires one application to listen/process multiple Kinesis streams. As the stream names are static, could even provide the stream names as part of the configuration itself.
The issue:
As the python client interacts with the Java Multilangdaemon, we have the ability to provide the configuration file to the MultilangDaemon. Yet :
streamName
in properties file. This field is expected to be a string.streamName
to the Multilangdaemon.How can we configure the KCL Python to use multiple stream? Thank you!