NerdWalletOSS / kinesis-python

Low level, multiprocessing based AWS Kinesis producer & consumer library
Other
118 stars 50 forks source link

Missing handling of unsuccessfully processed records #25

Open neongazer opened 4 years ago

neongazer commented 4 years ago

https://github.com/NerdWalletOSS/kinesis-python/blob/2bf355173de05a38f685df0f5a6000cff9a2076a/src/kinesis/producer.py#L126

By default, failure of individual records within a request does not stop the processing of subsequent records in a boto3 kinesis client put_records() request. This means that a returned records array from put_records() includes both successfully and unsuccessfully processed records. This library is missing detection of unsuccessfully processed records and handling in a subsequent call.