Look into pagination options for ES calls from webapp

aws-solutions / content-analysis-on-aws

As of August 30, 2023, this AWS Solution is no longer available. Existing deployments will continue to run. The functionality provided by Content Analysis on AWS will be superseded with functionality in Media2Cloud on AWS and Content Localization on AWS. We encourage you to explore these solutions.

https://aws.amazon.com/solutions/implementations/aws-content-analysis/

Apache License 2.0

96 stars 47 forks source link

Look into pagination options for ES calls from webapp #62

Closed brandold closed 1 year ago

brandold commented 4 years ago

This isn't part of your change. I noticed we are limiting the number of hits to return from Elasticsearch to 10000. If query results are truncated, we should indicate that to the viewer in the UI. I don't see that Elasticsearch _search API has a clear indication that results are truncated in the response. I think we should consider using the pagination ("scroll") in ES for our queries.

Originally posted by @aburkleaux-amazon in https://github.com/awslabs/aws-media-insights-engine/pull/76

ianwow commented 4 years ago

The limitation is 10,000, not 1000. Related to aws-solutions/aws-media-insights-engine#93

aburkleaux-amazon commented 4 years ago

I think we should also consider how much memory is being used by the front end to display the results and keep it in a certain bounds.