NationalMuseumAustralia / Collection-API

The public web API of the National Museum of Australia
10 stars 0 forks source link

Search doesn't tell you how many records its returning #6

Closed f27wood closed 6 years ago

f27wood commented 6 years ago

Search doesn't tell you how many records its returning which makes pagination difficult - this wouldn't be an issue if we do infinite scroll or something similar but if you want people to make their way through a result set its problematic.

f27wood commented 6 years ago

I believe this is a known issue which will be addressed in the public full version?

Conal-Tuohy commented 6 years ago

A feature has been added to report the total number of hits in an HTTP response header (Result-Count), along with a Link header with rel=next pointing to the next page of results. e.g.

Connection:Keep-Alive
Content-Type:application/json
Date:Mon, 23 Apr 2018 08:21:13 GMT
Keep-Alive:timeout=5, max=100
Link:<object?text=bark&offset=100>; rel=next
Result-Count:2654
Server:Apache-Coyote/1.1
Transfer-Encoding:chunked

API docs need updating to reflect the new functionality.

f27wood commented 6 years ago

Great!

I believe this would this meet Trove's need below. Is that right?

• Harvesting: Our harvester needs some way of determining how many records are in your API feed – so it can paginate through 100 records at a time and know when it’s reached the end. An example of this from another API feed can be seen here – they’ve given us the total record count in the field titled ‘found’: https://www.awm.gov.au/direct/data.php?q=type:Art&count=50 Are you able to include a total record count in your API that we can use to do this?

Conal-Tuohy commented 6 years ago

Yes I think so! Could you maybe direct the Trove person to this issue, so they can comment directly?

melGH commented 6 years ago

Hi Conal & Fiona Have checked with Julia & looks like the http response-header solution won't work for Trove. We need the result count or total record count to be within the json response - something like this example from NGV: http://www.ngv.vic.gov.au/custom/espa/collection/works

"took":2,"timed_out":false,"hits":{"total":69815,"hits":[{"_index":"collection_v2","_type":"works","_id":"98522","_score":1,"_source":{"artwork_id":"98522","title":"Dress and belt",