Adding Cursor Pagination to the gem by the Search Export Results

[KevinPedroza]

The Zendesk Support, Talk, and Help Center APIs have several endpoints that return lists of items, such as tickets, users, or articles.

For performance reasons, the APIs do not return large record sets all at once. Instead, they break up the results into smaller subsets and return them in pages. The number of items per page varies by the endpoint. For example, the tickets and users' endpoints return 100 items per page while the articles endpoint returns 30 items per page.

Zendesk has two methods of paginating through resources:

Offset pagination (previously referred to as just pagination). This method is supported by all resources. See Paginating through lists using offset pagination for detailed instructions. Cursor pagination. This method is supported by most resources, such as tickets and users, and will be progressively introduced to all resources. See Paginating through lists using cursor pagination for detailed instructions.

Zendesk recommends using cursor pagination instead of offset pagination where possible. Cursor pagination provides greatly improved performance when retrieving extremely large recordsets.

By the way, the offset pagination as far as we know will be active till September 15, 2021.

Offset Pagination

By default, most list endpoints return a maximum of 100 records per page. You can change the number of records on a per-request basis by passing a per_page parameter in the request URL parameters. Example: per_page=50. However, you can't exceed 100 records per page on most endpoints.

When the response exceeds the per-page maximum, you can paginate through the records by incrementing the page parameter. Example: page=3. List results include next_page and previous_page URLs in the response body for easier navigation:

Offset Pagination Documentation

Cursor Pagination

To use cursor pagination, include the page[size] parameter in the request parameters. This parameter is also used to specify the number of items to return per page. Most endpoints limit this to a maximum of 100. See the API documentation for the specific resource.

Use the URL in the links[next] property to retrieve the next page of results. Using the above example, make a request to https://example.zendesk.com/api/v2/tickets.json?page[size]=100&page[after]=xxx to retrieve the next page of results.

Repeat the above steps until the meta[has_more] property is false. This indicates there are no further records and you should stop paginating.

Cursor Pagination Documentation

Search Export Results

Exports a set of results. See Query basics for the syntax of the query parameter.

This endpoint is for search queries that will return more than 1000 results. The result set is ordered only by the created_at attribute.

The search only returns results of a single object type. The following object types are supported: ticket, organization, user, or group.

You must specify the type in the filter[type] parameter. Searches with type in the query string will result in an error.

Export Search Results Documentation

Solution

As far as we needed, we had to implement the search method but using the cursor pagination. Using the default search method would not allow us to use the cursor pagination only the offset. So, we decided to include the Search Export which includes the cursor pagination for our requirements. Due to this, we included some changes related to this.

[alextrueman]

Good time of day guys, do you plan to merge this PR? 👀

It would help us a lot 🙇🏻

[KevinPedroza]

Hey @alextrueman for sure, I just need someone to approve it!

[rvalenciano]

@alextrueman in the meantime I guess you can fork this gem and merge it in your canary so you can use the changes of this gem.

[snatchblock]

Hey @KevinPedroza ,

I apologise for the delay in approving the changes. Its all done now, and am ready to approve more changes if any needed here.

Please tag me whenever/whereever needed

[KevinPedroza]

Hey @snatchblock! No problem, for sure! Let me know when it is possible to merge it.