Request to migrate library from Offset Based Pagination to Cursor Based Pagination

[gkatechis]

Zendesk currently offers both offset-based pagination (OBP) and cursor-based pagination (CBP) for most API endpoints. CBP offers many performance advantages over OBP and will be the primary method of pagination offered at Zendesk in the near future. In an effort to provide a better, faster experience for our customers we are taking steps to encourage a transition to CBP – starting with the introduction of tighter limits on high-offset OBP requests.

What is changing? Beginning on August 15, 2023, OBP requests over the first 100 pages (10,000 records) will result in an error message: “400 Bad Request.” In order to request data sets larger than 10,000 records, customers will need to transition to CBP. OBP requests for fewer than 100 pages will not be affected, nor will requests made using the CBP model.

Why is Zendesk making this change? We are making this change in line with our previous announcement recommending adoption of CBP and to encourage customers to make the switch as soon as possible. Not only is cursor-based pagination faster and more efficient for our customers, it also puts less strain on Zendesk infrastructure and increases stability and reliability. As a first step, we are limiting offset-based pagination capabilities at a relatively high level, so that the transition can take place with minimal disruption to current API usage.

What do I need to do? Zendesk strongly encourages library maintainers to make the transition to cursor-based pagination before August 15, 2023. If you decide to not transition your library please add documentation in regards to the 100 page OBP limit to your library. If we have not heard from you in regards to this change prior to August 15, 2023. We will be removing our reference to your library from our documentation and will consider it abandoned / unmaintained.

[gkatechis]

Hi again from Zendesk! I wanted to follow up on this because it looks like the changes haven't been pushed yet. As a reminder, on August 15, 2023, OBP requests over the first 100 pages (10,000 records) will result in an error message: “400 Bad Request.” In order to request data sets larger than 10,000 records, you will need to transition to CBP.

[devrvinwebinarinc]

ff.

[gkatechis]

@devrvinwebinarinc Apologies for the delay...was that meant for me?

I also wanted to drop another update to make sure that the above has been seen. Starting last week, you may have begun seeing the results of the change, so make sure to address this in the near future!

[devrvinwebinarinc]

@devrvinwebinarinc Apologies for the delay...was that meant for me?

I also wanted to drop another update to make sure that the above has been seen. Starting last week, you may have begun seeing the results of the change, so make sure to address this in the near future!

Ow sorry I was just following up also since I am using this bridge on one of my sites.

[PierreGranger]

You can use the cursor based pagination with the actual library (3.8 and probably most previous too).

You have to use a particular syntax to make it work, here is an example before / after switching to cursor pagination :

Before :

$tmp = [];
$next_page = 1;
$secureLimit = 0;
while ($next_page !== null && $secureLimit++ < 100) {
    $result = $this->client->users()->findAll(['page' => $next_page++]);
    foreach ($result->users as $c) {
        $tmp[$c->id] = $c;
    }
    if ($result->next_page == null) {
        $next_page = null;
    }
}
return $tmp;

After :

$tmp = [];
$last_page = null ;
$secureLimit = 0 ;
while ($secureLimit++ <= 500) {
    $parameters = ['page[size]' => 100] ;
    if ($last_page != null && isset($last_page->meta->after_cursor)) {
        $parameters['page[after]'] = $last_page->meta->after_cursor ;
    }
    $result = $this->client->users()->findAll($parameters);
    foreach ($result->users as $c) {
        $tmp[$c->id] = $c;
    }
    if (! isset($result->meta->has_more) || $result->meta->has_more !== true) {
        break ;
    }
    $last_page = $result ;
}
return $tmp;

I'm not saying this code is particularly clean, it's just to give an example of the syntax you can use. I couldn't make it works with ['page' => ['size' => 100]] as i was expected.