stac-utils / stac-server

A Node-based STAC API, AWS Serverless, OpenSearch
MIT License
68 stars 27 forks source link

When too many collections are specified, the OpenSearch search url gets too long and causes an error #770

Open philvarner opened 4 months ago

philvarner commented 4 months ago

When searching, the collections parameter is appended to both the OpenSearch url (e.g., /coll1,coll2,coll2/_search) and to the search body as a terms predicate on the collection document attribute. These should be redundant, as all items in each index should have a collections value that's the same as the index. There is a small performance benefit to restricting the indicies, but this apparently causes an error when too many (~60) are specified.

The work here should be to not include the index names in the path if they are all greater than some limit, tbd, and instead simply treat it as an all-index search as if no collections had been specified (without the collections index, etc).

Exception is:

{"level":"error","message":"Response Error","meta":{"body":"<html>\r\n<head><title>414 Request-URI Too Large</title></head>\r\n<body>\r\n<center><h1>414 Request-URI Too Large</h1></center>\r\n</body>\r\n</html>\r\n","headers":{"connection":"close","content-length":"142","content-type":"text/html","date":"Thu, 16 May 2024 21:56:58 GMT","server":"awselb/2.0"},"meta":{"aborted":false,"attempts":0,"connection":{"_openRequests":0,"deadCount":0,"headers":{},"id":"https://URL/","resurrectTimeout":0,"roles":{"data":true,"ingest":true},"status":"alive","url":"https://URL/"},"context":null,"name":"opensearch-js","request":{"id":3,"options":{},"params":{"body":"{\"query\":{\"bool\":{\"must_not\":[],\"filter\":[{\"terms\":{\"collection\":....