Auto paginator feature

[kdubuc]

Hey !

Do you are interested by an Auto Paginator Feature ?

This is a generator that can be used to navigate across all results over all pages. When page boundaries are encountered, the next page will be fetched automatically for continued iteration.

This can be seen in Stripe SDK for example (https://stripe.com/docs/api/pagination/auto?lang=php).

This can be useful when Pagerfanta is connected with a remote API, to seamless retrieve a large collection.

I can work on a PR if you want.

[mbabker]

Do you have an example of what this API would look like?

At first, I'm not really seeing how this would really work well. There's already #30 which kind of fits into the automatic pagination design of Stripe's API but would focus on the more traditional database queries. And for a generic "use this adapter to automatically deal with pagination from any remote API" adapter design, I don't know how well that would work because pagination is going to be API dependent.

You can already kind of do this using the PagerfantaInterface, you'd just need your own iterator to deal with it.

$pager = Pagerfanta::createForCurrentPageWithMaxPerPage(
    new NullAdapter(1000),
    1,
    100,
);

autopager:
foreach ($pager->getCurrentPageResults() as $item) {
    // Do the things
}

if ($pager->hasNextPage()) {
    $pager->setCurrentPage($pager->getNextPage());

    goto autopager;
}

// All the data is processed

[kdubuc]

And for a generic "use this adapter to automatically deal with pagination from any remote API" adapter design, I don't know how well that would work because pagination is going to be API dependent.

The autoPagingIterator would use the power of the Pagerfanta pagination system (detect if more pages are present, select next page etc ...). And, the API / SQL Query / whatever pagination system would be managed in Adapter.

Do you have an example of what this API would look like?

Here we go !

Pagerfanta.php (I dont't fully tested it) :

/**
 * Iterate through large lists of resources without having to manually perform the requests to fetch subsequent pages.
 */
public function autoPagingIterator() : Generator
{
    while (true) {
        foreach ($this->getCurrentPageResults() as $item) {
            yield $item;
        }

        if (!$this->hasNextPage()) {
            break;
        }

        $this->setCurrentPage($this->getNextPage());
    }
}

This can be used to handle large collection easly.

Imagine this scenario : A remote service (DB, API ...) can served only 10 items per page. You need to retrieve all items (like 1000) to do a thing with. You must write an Adapter to connect on the remote service, and plug into PagerFanta instance. At this point, you need to write a do/while loop to manually retrieve all items. The autoPagingIterator do this for you !

Example :

$paginator = new Pagerfanta(new RemoteApiAdapter($api_filters, ...));

foreach($pagerfanta->autoPagingIterator() as $item) {
// do things
}

[mbabker]

If it's going to be a simple method like that, seems fine to me. I'm not sure how much use it might actually have since most of the API design here is opinionated for a "serve a single page of paginated data" workflow, but I don't think having this for more advanced use cases hurts much either.

[kdubuc]

I'm not sure how much use it might actually have since most of the API design here is opinionated for a "serve a single page of paginated data" workflow

Mainly to iterate through a large collection and avoid big memory consumption. You can encounter the same use case in Doctrine ORM https://www.doctrine-project.org/projects/doctrine-orm/en/2.14/reference/batch-processing.html#iterating-results.

I will fork Pagerfanta, add the code and tests, and submit you a PR !

Thank you !