search

simolus3 / drift

Drift is an easy to use, reactive, typesafe persistence library for Dart & Flutter.
https://drift.simonbinder.eu/
MIT License
2.55k stars 365 forks source link

Paginated QueryStream #412

Open szotp opened 4 years ago

szotp commented 4 years ago

I've noticed that watch() is not very well optimized for larger fetches because on every update it fetches all data and maps it into objects.

A typical case for larger fetch is displaying items in a very long list: chat messages, mails, history, etc.

So perhaps moor should provide something like NSFetchedResultsController from CoreData - as users scrolls list / grid, it would query further pages, and probably discard previous ones.

I'm not sure what would be the best way to do this, maybe have something that maintains QueryStream for current page, and then recreates it when user scrolls further.

Abdktefane commented 3 years ago

i have the same use case i want to observe data from table that have a lot of data so i load the first 20 and append them to list and so on, but when first values changed in database my values in list not updated and that normal because it's not a stream . any helpful idea to apply pagination with moor and stream without loading all data please @simolus3 @szotp ?

szotp commented 3 years ago

I have a solution for this, I'll post it during the weekend. Basically you have to watch first page of data and adjust your big array if something gets inserted or deleted.

Abdktefane commented 3 years ago

i'll be waiting for you, please do not forget and thank you .

szotp commented 3 years ago

Here it is: https://github.com/szotp/moor_fetcher/blob/master/lib/moor_fetcher.dart You can copy the code or import is as package.

jarjut commented 2 years ago

@szotp is there any updated code for this issue? thankyou

deimantasa commented 1 year ago

Curious, if there is already any implementation to help above use-case?

deimantasa commented 1 year ago

To give an update for it and how I managed to solve it - assume you've list of items which you'd like to paginate over and listen for its changes: Assuming you locally have List<T>.

  1. Get length of List<T> and based on it, get the offset for pagination
  2. Check if there are any more T in the DB based on the offset. 2a. If there are not - terminate querying 2b. If there are, get more T based on the offset and page size
  3. Iterate through new List<T> and attach watch query to each item to observe the changes 3.1. If item is null - remove from the local list otherwise replace in the local list
  4. Add all List<T> from DB to local list

This way we do not need to keep track of pagination as it will be dependant on the length of the items in the local list. Further, only these item changes are being listened to.

Let me know if anyone finds it beneficial - can put up a full implementation of it.

DennisMuchiri commented 1 year ago

@deimantasa Hi please post cose sample example for how you solved thesw