v4 - How do I get paged items using async iterator and break only when ALL items are retrieved?

[tandrasi]

What version of PnPjs library you are using

4.x

Minor Version Number

0.1

Target environment

SharePoint Framework

Additional environment details

Using SPFx 1.18.2 to create a webpart for SharePoint Online using standard SharePoint Authentication.

Question/Request

I'm having issues upgrading to v4 from v3 because I cannot figure out how to use the Async Iterator pattern to break/close the loop only when all items are retrieved.

We have a shared SPFx library with a function to retrieve list items. This used the "getPaged()" function and we simply looped through the collection while "hasNext" was true:

while (itemCollection.hasNext) {
    itemCollection = await itemCollection.getNext();
    results = results.concat(itemCollection.results);
}

This stopped the code from running further and it allowed us to return all items at once when everything was fetched. This is a generic library we have for all our apps, and we do post processing of the items prior to return.

I've tried using the Async Pattern in the docs but cannot figure out how to delay return until all items are gathered:

//using async iterator in combination with top() to get pages of items in chunks of 10
for await (const items of sp.web.lists.getByTitle("BigList").items.top(10)) {
  console.log(items); //array of 10 items
  break; // <---- HOW DO I BREAK/CLOSE THE ITERATOR ONLY WHEN ALL ITEMS ARE RETRIEVED?
} 

The docs show that we can access the iterator directly from the collection, but I don't understand what/where to put this and how to use this:

const iterator = collection[Symbol.asyncIterator]();

I saw other docs about a Paging Helper but the AsyncPager class doesn't exist in pnp/js.

[juliemturner]

The answer is you don't break if you want all the items. This will get all the items in batches of 10.

let allItems = [];
for await (const items of sp.web.lists.getByTitle("BigList").items.top(10)) {
  allItems = allItems.concat(items);
} 

[tandrasi]

I had a reply here in which I was using Array.concat incorrectly, but yes, what you proposed does indeed work. Many thanks.

[github-actions[bot]]

This issue is locked for inactivity or age. If you have a related issue please open a new issue and reference this one. Closed issues are not tracked.