search

sixthnormal / clj-3df

Clojure(Script) client for Declarative Dataflow.
Eclipse Public License 2.0
328 stars 13 forks source link

Pagination #31

Open metasoarous opened 5 years ago

metasoarous commented 5 years ago

Is pagination of queries on your radar at all?

comnik commented 5 years ago

Anything involving sorting is tricky, I haven't thought much about it, and it's not at the top of the priority list right now, but here goes.

We do not want 3DF to maintain explicit order in the general case, because most input diffs (to a list of n records) can be expected to produce O(n/2) output diffs (to every connected client!). It seems a compromise would be for the backend to perform something like a top-k by the desired metric and then assign a page number to every record. Within a page, clients are responsible for presenting them in sorted order.

For pages of size k this should cut down the expected number of output diffs significantly (something like n/k * probability of any individual record moving between pages?).

How does that sound?