[API] /search : Non contiguous annotations result with ?search_after

[kael]

Describe the bug When /search'ing for annotations with the ?search_after search parameter, the server returns non contiguous annotations.

But the bug seems to happen only on the first batch (unclear yet).

Fetching after the date of the last annotation returned from the previous search returns results starting with non contiguous annotations, like if several annotations were skipped.

(Please, see the linked gists for full annotations samples).

To Reproduce

1. Search for 1 annotation.

   • https://api.hypothes.is/api/search?limit=1

     {
     "id": "nBXu2lLQEeyV2lNeCgRXyA",
     "created": "2021-12-01T18:00:46.426922+00:00",
     }

2. Search for 2 annotations.

   • https://api.hypothes.is/api/search?limit=2

     
     {
     "id": "nBXu2lLQEeyV2lNeCgRXyA",
     "created": "2021-12-01T18:00:46.426922+00:00",
     }

{ "id": "hpN4_FLQEeywQ98rkS4JVw", "created": "2021-12-01T18:00:10.429872+00:00", }

3. [Search for 1 annotation after the first annotation creation date](https://gist.github.com/kael/27ceeee9d714c0b4ffafd6c900570148) (from 1st example): the next returned annotation is not the same as the second annotation in the second example:
- https://api.hypothes.is/api/search?search_after=2021-12-01T18:00:46.426922+00:00&limit=1
```json
{
      "id": "YCs7TFI5EeyqEndNCfG9Fw",
      "created": "2021-11-30T23:58:12.125455+00:00",      
}

// Should be
{
      "id": "hpN4_FLQEeywQ98rkS4JVw",
      "created": "2021-12-01T18:00:10.429872+00:00",
}

Expected behavior Paging through annotations with the ?search_after parameter should return contiguous annotations.

[robertknight]

You need to encode your query parameters correctly. Compare this command without correct encoding:

curl 'https://api.hypothes.is/api/search?limit=1&sort=created&order=desc&search_after=2021-12-01T18:44:21.321350+00:00'

With this command which does use correct URI encoding:

curl 'https://api.hypothes.is/api/search?limit=1&sort=created&order=desc&search_after=2021-12-01T18%3A44%3A21.321350%2B00%3A00'

I encoded the search_after parameter here using encodeURIComponent('2021-12-01T18:44:21.321350+00:00') in Node, but use the standard URL generation APIs for your programming language.

I also recommend to specify the sort field and sort order explicitly when using search_after.

[kael]

You need to encode your query parameters correctly. Compare this command without correct encoding:

Indeed, it works with encodeURIComponent.

Thanks for the tips ! And sorry for having opened this ticket.

P.S.: I changed the fetch client on my side and forgot to set encodeURIComponent back.