Open NatElkins opened 6 years ago
I think that getting millions of items need to be done with paging so why use AsyncSeq and not create function that get page size and callback function?
I made a start on paging support a year or so ago but never finished. It seems a more flexible approach as you’d often want to paginate data via an API or a user interface. I can have another go at this if it seems useful?
You could certainly then wrap a paginated query in AsyncSeq
(and I think this is worthwhile for admin-type scripts).
See @bartelink’s comment here: https://github.com/fsprojects/FSharp.AWS.DynamoDB/pull/16#issuecomment-1619621391 - I think there’s value in implementing scans & queries using TaskSeq
, it may need some thought around the API design. I wouldn’t be fundamentally opposed to returning a TaskSeq
as standard from all unbounded scans/queries
I have a use case where I may need to scan a table for millions of items. I am unable to allocate an array that large. What I really want is to be able to operate on it like a stream. Would you accept a PR adding an overload that returns an AsyncSeq instead of an array? It would require taking a dependency on https://github.com/fsprojects/FSharp.Control.AsyncSeq.
Thank you for all your hard work!