dlr-eoc / prosEO

prosEO – A Processing System for Earth Observation Data
GNU General Public License v3.0
14 stars 1 forks source link

prosEO REST API: Restrict number of records retrievable with any single request #169

Open tangobravo62 opened 1 year ago

tangobravo62 commented 1 year ago

Currently it is possible to e. g. query for all products in the metadata database (via the Ingestor "products" REST service). This may result in a huge amount of records being returned, blocking the service in question for other work and possibly even exhausting its memory limits.

All REST APIs returning lists of objects shall therefore be restricted to a configurable maximum number of records they return (which means that also some method of paging must be implemented). This is already the case for the PRIP API, where in case of too large result sets the HTTP status code 429 (Too many requests) is returned.

For better manageability of the implementation sub-issues shall be created for the individual microservices.