Query.py scan does not support Pagination ?

[DzakirinMD]

Question Hey,

I saw inside pydruid/pydruid/query.py that pagination in select is supported, but scan does not support pagination. Will future release of pydruid scan support pagination as well ? since start of Druid 0.17 scan already supported pagination.

[paul-rogers]

More specifically, the Druid documentation says that the offset property is supported for the scan query, but the QueryBuider.scan() function does not. FWIW, the batchSize property is also not supported in pydruid.

There is a workaround, first build the query, then doctor it, then run it. Instead of:

query = client.scan(datasource=..., limit=20, offset=20)

Do it the long way:

from pydruid.query import QueryBuilder
qb = QueryBuilder()
scan = qb.scan({
    'datasource': ...
    'limit': 20,
})
scan.query_dict['offset'] = 40
query = client._post(scan)