Kinesis enhanced fan-out consumers

[gee-m]

Hey there, love the tool!

Just a feature suggestion in case more people are interested than just me - it would be cool to support kinesis enhanced fan-out consumers. As far as the research I've done, here's some notes hopefully that can help see clearer into this task

• It would require some change as we use GetShardIteratorWithContext but would need to use SubscribeToShard with SubscribeToShardEvents
• It could be a new input?
• Could use the VMWare kinesis library to make it easier
• Ability to either RegisterNewStreamConsumer or use an existing one (e.g. defined in terraform) would be cool

Here's some resources:

• AWS Go docs
• AWS Usage example (java)

[saholman]

This is something I'm going to need in the near future. @Jeffail are there plans to add this to the existing Kinesis input? If I implement a new consumer that supports Kinesis enhanced fan-out, is that something you'd be interested in merging back into the baseline?

[Jeffail]

@saholman yeah I'd be happy to take a PR, ideally something interacting with the AWS client libraries directly rather than via another dependency, but happy to take a look at anything.

[michael-pressburg-cko]

Are there any updates on this? @Jeffail