Open adrian-skybaker opened 2 years ago
Hello do you encouter this error also ?
time="2023-11-17T13:23:29Z" level=error msg="[kinesis 0] The partition key could not be found in the record, using a random string instead"
Hello do you encouter this error also ?
time="2023-11-17T13:23:29Z" level=error msg="[kinesis 0] The partition key could not be found in the record, using a random string instead"
You need to make sure that you actually have defined partition_key as a key in your log messages
I'm finding that I see significant numbers of duplicates if I hit throttling on the kinesis stream.
Obviously I realise I want to avoid throttling, but I'm wondering this is expected behaviour? For example, I would expect that even when batching, the plugin would only retry the failed parts of the batch.
If this is not expected, happy to provide more logging if that's helpful (below is warning level and above).
This is using amazon/aws-for-fluent-bit:init-2.28.1 .
Log sample: