amazon-archives / amazon-kinesis-connectors

Apache License 2.0
325 stars 191 forks source link

Implement ConstantRecordCountMemoryBuffer #73

Closed slam closed 5 years ago

slam commented 8 years ago

ConstantRecordCountMemoryBuffer is an implementation of the IBuffer interface. It ensures that records are periodically flushed with the same record count.

Flushing a constant number of records is essential to the proper handling of duplicates on the consumer end as documented here:

http://docs.aws.amazon.com/streams/latest/dev/kinesis-record-processor-duplicates.html