search

gbif / tech-docs

This is an integrated technical documentation site for GBIF.org
https://techdocs.gbif.org
0 stars 2 forks source link

Expand information for the limit API parameter #73

Closed nickynicolson closed 1 month ago

nickynicolson commented 6 months ago

Feedback on Occurrence API (Github source for this page)

Currently the API documentation just shows the following for the limit parameter:

Controls the number of results in the page. Using too high a value will be overwritten with the default maximum threshold, depending on the service. Sensible defaults are used so this may be omitted.

Given that the user is on the documentation page for a specific service (in this case occurrences) it would be helpful to know exactly what the defaults and maximum values are. The same generic text is shown on the API documentation for the registry.

CecSve commented 4 months ago

I believe the information you are looking for can be found in the text above the sections? E.g. for occurrence downloads:

Occurrence Download Limits

Occurrence downloads demand significant computational resources, and are monitored and limited according to the GBIF platform load. In order to avoid that downloads requested by a single user utilize most of the resources two rules have been set:

Download complexity limits:
    A download predicate may contain a maximum of 101,000 items (taxon keys, kingdom keys, phylum keys, catalogue numbers, occurrence ids and so on).
    A download predicate may contain a maximum of 10,000 points in any “within” predicate geometries.
Limits on incomplete (preparing, running) downloads:
    If the total number of incomplete downloads is fewer than 100, any single user can have no more than 3 incomplete downloads.
    If the total number of downloads is fewer than 1000 any single user may only have 1 download.

I think we opted to have the text higher up to cover more details.

MattBlissett commented 4 months ago

It's the other bit above:

In order to retrieve all results for a given search filter you need to issue individual requests for each page, which is limited to a maximum size of 300 records per page. Note that for technical reasons we also have a hard limit for any query of 100,000 records. You will get an error if the offset + limit exceeds 100,000. To retrieve all records beyond 100,000 you should use our asynchronous download service (below) instead.

On some other APIs (registry etc) the same bit of text applies to methods with different limits, but as this isn't the case for the occurrence search API I can change it.