search

bcgov / aries-vcr

Hyperledger Aries Verifiable Credential Registry (VCR) is a set of application level software components designed to accelerate the adoption of trustworthy entity to entity communications.
Apache License 2.0
78 stars 70 forks source link

OrgBook BC API returns a 400 Error #765

Closed amanji closed 9 months ago

amanji commented 9 months ago

It was reported that the (v3/v4)/search/topic endpoint returns a 400 Error when paging past a certain number of results.

For example: https://orgbook.gov.bc.ca/api/v4/search/topic?ordering=-score&q=abc&inactive=false&latest=true&revoked=false

Returns 541 results

The first 10 pages of results can be accessed:

https://orgbook.gov.bc.ca/api/v4/search/topic?inactive=false&latest=true&ordering=-score&page=10&q=abc&revoked=false

The result of the 10th page shows a URL to the next page available:

Image

Accessing the 11th page onwards returns a 400 Error. This is consistent regardless of the query used.

Image

swcurran commented 9 months ago

That’s a “feature” that @WadeBarnes added to stop bots from scraping the database.

Who reported the issue?

WadeBarnes commented 9 months ago

@swcurran, Correct

esune commented 9 months ago

Closing as this is a known behaviour.

For context, the limit to query paging was introduced to avoid issues deriving from performing queries that fetch large data sets (unbounded) that would cause out-of-memory issues in the search engine.

References: