NerdWalletOSS / kinesis-python

Low level, multiprocessing based AWS Kinesis producer & consumer library
Other
118 stars 50 forks source link

Support using enhanced fan-out for consumer #32

Open givenday opened 1 year ago

givenday commented 1 year ago

Hi! First of all thanks for the work done on this library. Definitely using KCL is not ideal with dependency on java MultiLangDaemon...

Wanted to know if there are plans to use enhanced fan-out for the consumers? I noticed it is using get_records() instead of subscribe_to_shard(). This would have limitation on the throughput of 2mb/s shared among consumers per shard. https://docs.amazonaws.cn/en_us/streams/latest/dev/building-enhanced-consumers-api.html

Do you have plans to migrate to using enhanced fan-out for the consumers?