amazon-archives / amazon-kinesis-connectors

Apache License 2.0
326 stars 190 forks source link

Pass shard ID into IEmitter.emit() #72

Closed slam closed 5 years ago

slam commented 8 years ago

For the S3Emitter, it is necessary to include the shard ID in the S3 filename in order to properly handle duplicate records on the consumer side as recommended here:

http://docs.aws.amazon.com/streams/latest/dev/kinesis-record-processor-duplicates.html