Is it possible to use multiple java app consumers for single kinesis stream using single dynamodb table for checkpoint information(Using KCL2)

[rupeshshah4706]

We have been successfully using multiple java app consumers(multiple kinesis applications) for single kinesis stream using dynamodb table per consumer for checkpoint information. But now we have a use case where we have to give single dynamodb table as parameter to KCL2 client.

We are going to deploy java service on EC2 using auto scaling group. Multiple instances will act as a separate consumers for same kinesis stream, and we also have to give single dynamodb table as parameter to KCL2 client.

We are deploying our entire infrastructure using terraform, and I think it is not good idea to create dynamodb table per consumer for storing checkpoint information.

If its possible then It will be very helpful if you point me to KCL2 client creation code snippet.

[eoliphan]

I'm fairly certain that this does not work, as I "tried" this accidentally with a misconfiguration. The KCL code creates a table for a given KCL "application" (which can have N number of workers), and per my understanding of the docs, there's a 1-1 relationship between an KCL app and a stream, attempting to point a single app (table) to multiple streams seems to confuse things for the KCL client. So you will end up with a table for each distinct KCL app

[ShreySharma1]

@rupeshshah4706 I am facing same use case, can you tell me if you reached any conclusion in how to tackle this ?