search

awslabs / amazon-kinesis-producer

Amazon Kinesis Producer Library
Apache License 2.0
399 stars 331 forks source link

KPL Warning Message: PutRecords processing time is taking longer than 500 ms to complete #429

Open lakshmikanthpadala opened 2 years ago

lakshmikanthpadala commented 2 years ago

We are using KPL to post messages to Kinesis Stream using default KinesisProducerConfigurations. We have been observing the following warning messages intermittently.

[processing_statistics_logger.cc:126] PutRecords processing time is taking longer than 500 ms to complete. You may need to adjust your configuration to reduce the processing

KPL Version 0.14.6

As per our observation from CloudWatch metrics, we don't see any throttled requests. Also no retries on the requests. We are trying to see if there is anyway to fine tune KPL and find what is causing this warning.

We would like to know the following.

  1. How can we find what is causing this sudden warning?
  2. Are there any specific metrics which we can tune?
  3. Are there any tools which can help in finding the root cause and fine tune KPL configurations ?

Please let us know.

kivaturilucid commented 2 years ago

We are facing the same issue. I am surprised this issue is not addressed for years now. Looks like there is no active support for this library

VladimirPchelko commented 2 years ago

maybe this https://github.com/awslabs/amazon-kinesis-producer/issues/48

Too large value of OutstandingRecordsCount causes native part (see C++ code) of KPL to consume too much CPU time. It seems that this "bug" has existed since the appearance of this library - and no one is going to fix it...

maxdbn commented 1 year ago

I'm having the same warning message with the default configuration. Is there anything that needs be configured differently?