Default pagination.paginate function doesn't check for empty string link headers

joshua-leyshon-canva commented 3 years ago

Describe the bug

Node.js version: v12.22.1
OS & version: MacOS Big Sur, Ubuntu 20

Sometimes (e.g. with Buildkite REST API), a response can contain a link header that is an empty string. The default paginate logic (see just below) doesn't handle this case and will throw when it tries to parse the string. This means we need to add our own paginate logic that is almost identical to the default.

Current default function: https://github.com/sindresorhus/got/blob/06a2d3d7d8d4fcc6898b6364d1a18ca1d407092b/source/core/options.ts#L677

A fix:

if (typeof response.headers.link !== 'string' || response.headers.link === '') {
  return false;
}

Actual behavior

got throws when parsing an empty string link header.

Expected behavior

The default pagination.paginate function should return false when the link header is an empty string.

Code to reproduce

const res = got.paginate.all('https://somewebsite.com/that-returns-empty-link-header');
// An example endpoint (requires Buildkite authentication):
// "https://api.buildkite.com/v2/organizations/{org.slug}/pipelines/{pipeline.slug}/builds/{build.number}/jobs/{job.id}/artifacts"

Checklist

[x] I have read the documentation.
[x] I have tried my code with the latest version of Got.
[ ] I have tried my code with the latest version of Node.js.

joshua-leyshon-canva commented 3 years ago

Note that I have separately raised the issue of returning an empty link header with Buildkite as well, but I figured we could also improve the default paginate function here 🙂

sindresorhus commented 3 years ago

What does the spec say about empty link? If it's a spec violation, we should at least emit a warning, even if we decide to gracefully handle it.

joshua-leyshon-canva commented 3 years ago

Good question. This doc is the closest spec I found, which outlines what a link string should look like, but doesn't explicitly say anything about an empty link. Might be up to you how to handle this case 😄

szmarczak commented 3 years ago

Actually might be empty. The specification defines #link-value, which means from 0 to infinity occurrences.

https://datatracker.ietf.org/doc/html/rfc1945#section-2.1

joshua-leyshon-canva commented 3 years ago

Hey @sindresorhus @szmarczak I opened https://github.com/sindresorhus/got/pull/1768 to fix this issue 🦆

sindresorhus / got