[RRFC] Support sending custom request headers to an npm registry (`$MY_AUTH_HEADER: $VALUE`)

TysonAndre commented 2 years ago

Motivation ("The Why")

I'd want to be able to set custom request headers in order to be able to use npm registries behind a proxy that requires a specific header key and value (Another use case might be custom caches/registries (for multiple programming languages) that require a header that isn't Authorization - this isn't a use case I've had, though)

Proxies that have a mandatory HTTP custom header to authenticate are an alternative for hosting a private/internal registry, which can be used without needing a VPN

When a proxy is used to connect to a registry, it may have no way to accept an Authorization: HTTP header, especially if the applications it proxies use the Authorization for basic auth or oauth

The motivation of this is similar https://github.com/npm/npm/issues/18583

Example

e.g. //my.private.registry.host.com/registry/path/:_customAuthHeader = My-Auth-Header: abcXYZ123:/== in a config file used by the npm cli would set req.headers['My-Auth-Header'] = abcXYZ123:/== (not familiar with whether this expects escaping or not, it should do what the _auth header does)

How

Current Behaviour

Only a few hardcoded headers are supported, as shown in the snippet: authorization:, and some npm-* headers

https://github.com/npm/npm-registry-fetch/blob/42d605cb00986c6775e6ec1732b114e065266ae9/lib/index.js#L211-L242

Desired Behaviour

Add support for adding custom headers through the .npmrc file

e.g. //my.private.registry.host.com/registry/path/:_customAuthHeader = My-Auth-Header: abcXYZ123:/== in the .npmrc file used by the npm cli would set req.headers['My-Auth-Header'] = abcXYZ123:/== for requests to //my.private.registry.host.com/registry/path/ (not familiar with whether this expects escaping or not, it should do what the _auth header does)
Optionally, this header could be repeatable, though I expect to only need one header set for the vast majority of use cases

Known http headers that have existing non-auth meanings or the npm headers that are already set may make sense to exclude (and emit a warning), with a case-insensitive string comparison (e.g. Content-Type, authorization, user-agent, etc.)

This could be limited to [a-zA-Z0-9_-] for the header key, and emit an error for other headers, to prevent sending invalid/ambiguous values, spaces, etc in the header key

Key names found in Object.prototype such as __proto__ and toString could also be rejected just in case a custom server itself was buggy or an application using the npm-registry-client library other than npm was buggy
The npm-* header prefix for unknown headers is something that I don't need - I'm fine with forbidding that entire prefix initially and leaving any RRFC for that to work to anyone who has a use case for those

I'd imagine this be similarly able to be scoped to a registry, similar to the always-auth/_authToken settings.

The key and value could be split on the first '=' and trimmed, then set the same way as other values in the getHeaders function

https://github.com/npm/npm-registry-fetch/blob/42d605cb00986c6775e6ec1732b114e065266ae9/lib/index.js#L211-L242

References

The motivation of this is similar https://github.com/npm/npm/issues/18583

I'd filed https://github.com/npm/npm-registry-fetch/issues/145 before discovering the rfcs repo (CONTRIBUTING.md in npm/npm-registry-fetch was a 404)

TysonAndre commented 2 years ago

https://github.com/npm/rfcs/pull/138 - This seems to be related, but was last updated in 2020. (I'd only searched the issue tracker and had missed the PR)

https://github.com/npm/rfcs/pull/138#issuecomment-729831382 mentions a workaround for headers

EDIT: the headers section works in npm 6 (including 6.14.17), but does not work in npm 8, probably because of the refactoring fixing the unintentional feature

(using nc -l 12345 to see what raw headers were sent)

registry = http://127.0.0.1:12345
[headers]
my-xyz = xyz

https://docs.npmjs.com/cli/v6/configuring-npm/npmrc doesn't document headers and it won't work in npm 8, so it seems unsupported.

If headers ends up being supported and documented, though, perhaps it could start warning or emitting errors for some of the cases I'd mentioned

This could be limited to [a-zA-Z0-9_-] for the header key, and emit an error for other headers(content-type, authorization, user-agent, etc), to prevent sending invalid/ambiguous values, spaces, etc in the header key

TysonAndre commented 2 years ago

It seems like npm 9 will be dropping support for global configs entirely for _auth, _authToken, always-auth, etc, and only allowing those configs to be scoped to a given registry.

e.g. https://github.com/npm/cli/pull/5696

So maybe there should be no global [headers] section in an existing or new RFC to do this, and only a section such as this:

[//my.registry.com/my/prefix/path:headers]
my-xyz = xyz

nlf commented 1 year ago

So maybe there should be no global [headers] section in an existing or new RFC to do this, and only a section such as this:
[//my.registry.com/my/prefix/path:headers]
my-xyz = xyz

if this is something we do, this is exactly how it would need to work. allowing these headers to be defined at the top level introduces the same risks as having credentials at the top level - they could be leaked to a registry you don't want them going to.

atamayoz commented 6 months ago

Is there any news about this feature?

npm / rfcs