search

aws-solutions / content-analysis-on-aws

As of August 30, 2023, this AWS Solution is no longer available. Existing deployments will continue to run. The functionality provided by Content Analysis on AWS will be superseded with functionality in Media2Cloud on AWS and Content Localization on AWS. We encourage you to explore these solutions.
https://aws.amazon.com/solutions/implementations/aws-content-analysis/
Apache License 2.0
96 stars 47 forks source link

Elasticsearch write rejection #34

Closed ianwow closed 3 years ago

ianwow commented 3 years ago

I'm getting the following error in the Elasticsearch consumer lambda (line 811) for long videos that generated paged results from Rekognition.

Unable to load data into es: TransportError(429, '429 Too Many Requests /mielabels/_bulk')

The results in the UI are clearly missing paged results:

image
ianwow commented 3 years ago

Possible causes are pegging the python elasticsearch library to 7.13.4, or rev'ing Elasticsearch Service to 7.10.

Whatever the cause, I think we need to implement an exponential retry logic for bulk loads rather than just bailing out when bulk loads fail.

ianwow commented 3 years ago

Retries don't appear to help, but reducing maxPayloadSize does help.