Open LadaRudnitckaia opened 3 years ago
First finding: in the current version of {crul}
, only offset-based pagination is implemented in the Paginator
class:
#' Supported now:
#'
#' - `limit_offset`: the most common way (in my experience), so is the default.
#' This method involves setting how many records and what record to start at
#' for each request. We send these query parameters for you.
#' - `page_perpage`: set the page to fetch and (optionally) how many records
#' to get per page
#'
#' Supported later, hopefully:
#'
#' - `link_headers`: link headers are URLS for the next/previous/last
#' request given in the response header from the server. This is relatively
#' uncommon, though is recommended by JSONAPI and is implemented by a
#' well known API (GitHub).
#' - `cursor`: this works by a single string given back in each response, to
#' be passed in the subsequent response, and so on until no more records
#' remain. This is common in Solr
(see [here])https://github.com/ropensci/crul/blob/master/R/paginator.R#L27-L43)
As Kobotoolbox is using a link-header style pagination, we hence have to implement it by ourselves. However, the design of Paginator
might be useful/inspiring :)
cc @dimitri-mar @LadaRudnitckaia
Design and implement pagination for
GET
A reasonable requirement for kbtbr is to handle paginated server responses correctly, such that users to not silently receive only a partial response. This should actually be implemented before the first release, at best.
To that end, please
get
of theKobo
classTBD
Paginator
fromcrul
or write the code ourselves.