Pagination

[afandian]

Make sure pagination works to our satisfaction.

Definition of done:

• All resources that include pagination listed and documented, and the different sort orders available.
• Implementation should allow that in all cases pagination allows paging through a whole, large result set (e.g. 1 million items), with any search ordering.
• Integration tests include deep paging.
• Pagination documented in user docs.

[afandian]

Tasks: 1) List resources available in the ES branch. For each:

• URL
• Current pagination facility (offset+rows, cursor) and standard / max page size
• Approx number of items available - do we need to scroll?
• Sorting options available. 2) Make sure swagger and API error responses docs are correct / update to desired behaviour. 3) Close reading of the Elastic Search docs for cursors. 4) Decide on suitability of each method for each resource.
• Maybe some hard questions, like limiting pagination for query-ordered results?

[afandian]

Idea for spec after some investigation:

• TBC routes have scroll available.
• Not allowed to specify order and cursor at the same time (currently can but it does nothing).
• Allowed to do page + offset when ordering is applied, but we limit the depth for performance reasons.
• With cursors the return order can't be specified.
• Time-outs on cursors are explicit.
• Can't cross from one pool to another mid-page.

[afandian]

Resources up for discussion:

• works
• journals
• types
• cores
• funders