Docs/Samples: Looping through entire queryable

[christophwille]

The current sample is for interactive - how about a sample for headless where the entire purpose is to not bring in everything at once? (think huge tables that simply need to scanned)

[mrahhal]

Can you elaborate? What kind of sample are we talking about exactly? If by "not bring in everything at once" you mean columns, creating a dto isn't that much different.

[christophwille]

Roughly this:

int rows = 0;
do
{
  rows = 0;
  await foreach (var item in ctx.MyDbSet.Orderby(b => b.Id).Where(b => b.Id > lastId).Take(100))
  {
    // do something here
    rows++;
    lastId = item.Id;
  }
  ctx.ChangeTracker.Clear();
} while (rows == 100);

Maybe also thinking a bit about the PageIterator from MS Graph SDK 5.

[mrahhal]

Ah, you're talking about batching. There are very specific use cases for this and the way you would deal with doing some work in batches is pretty much the same in non KeysetPagination code (but instead of storing a lastId you would store the last entity). This is particularly simple because you'll only ever need a keyset containing the id for a scan.

But I do see that KeysetPagination is particularly efficient in such batched scanning operations compared to the classic method, so might be worth it to have a sample on this after all.

[skwasjer]

The example I provided for a different issue seems relevant here, even though I encapsulated it. It could also be rewritten into a convenient extension method btw without the MediatR sugar/encapsulation ;)

https://github.com/mrahhal/MR.EntityFrameworkCore.KeysetPagination/issues/58#issuecomment-2380072273