search

poissonconsulting / fwapgr

An R Client for BC Freshwater Atlas Web API
https://poissonconsulting.github.io/fwapgr
Other
8 stars 1 forks source link

Pagination? #47

Closed joethorley closed 3 years ago

joethorley commented 3 years ago

pagination would be really helpful - is there a way to elegantly do this in fwatlasbc?

sebdalgarno commented 3 years ago

first need total number of rows in query result (before result returned)

smnorris commented 3 years ago

easy to request a count via WFS: https://github.com/smnorris/bcdata/blob/master/bcdata/wfs.py#L119

Not sure about OGC API features / pg_tileserv

sebdalgarno commented 3 years ago

@dr-jts is it possible to just return the number of records from a query in pg-featureserv? this would be useful for pagination...

dr-jts commented 3 years ago

is it possible to just return the number of records from a query in pg-featureserv? this would be useful for pagination...

Not at the moment. Best that can be done right now is to provide a function taking a table name as a parameter and returning the count of records in the table.

The OGC API for Features spec does provide support for paging via a numberMatched property as well as next and prev links, but these are all optional. They're not implemented in pg_featureserv right now partly because they would impose extra overhead on queries (e.g. to return the number of records found a query would have to be run twice).

I guess the generation of these items could be controlled via a configuration option. Or more efficiently, it could be controlled on a per-request basis via an option flag (a query parameter or perhaps a HTTP header?).

smnorris commented 3 years ago

This reminds me of https://github.com/CrunchyData/pg_featureserv/issues/54 I wonder if a count estimate https://wiki.postgresql.org/wiki/Count_estimate could be 'good enough' for collections? For functions, yes - per-request does seem the way to go, that would be a big performance hit.