NPM_CONFIG_REGISTRY seemingly ignored on subsequent runs of `deno cache`

[webbower]

Deno 1.29.1 macOS 11.6.4

I added my first import for an npm: specifier and ran NPM_CONFIG_REGISTRY=https://registry.mycompany.com/path/to/destination/ deno cache --node-modules-dir to download a local copy through my company's Artficatory (public NPM is blocked) and it worked fine. However, when I run NPM_CONFIG_REGISTRY=https://registry.mycompany.com/path/to/destination/ deno cache ... again, I get BadCertificate errors with additional output showing that it seems to be trying to use the default NPM registry again.

Sending fatal alert BadCertificateg/<npm-package> (0/#)
Sending fatal alert BadCertificate
Sending fatal alert BadCertificate
Sending fatal alert BadCertificate
Sending fatal alert BadCertificate
Sending fatal alert BadCertificate
error: failed reading lockfile '/path/to/project/deno.lock'

Caused by:
    0: Error getting response at https://registry.npmjs.org/<dependency-package>
    1: error sending request for url (https://registry.npmjs.org/<dependency-package>): error trying to connect: invalid peer certificate contents: invalid peer certificate: UnknownIssuer
    2: error trying to connect: invalid peer certificate contents: invalid peer certificate: UnknownIssuer
    3: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

After some trial and error, I found that I could run the deno cache command again if I deleted the lock file (which oddly doesn't capture the registry URLs of the NPM packages).

Not captured in the above output as the first line because it gets overwritten by the error output is:

Download https://registry.npmjs.org/<dependency-package>

Which seems to confirm that it is falling back to the default registry. When I run the command after deleting the lock file, that first line of output shows the correctly overridden registry URL.

I did some digging into the Deno codebase to find how the NPM registry is set and why it would fail on subsequent runs of deno cache with NPM_CONFIG_REGISTRY set and I found this where the Registry URL is produced. I don't know Rust at all so figuring out where this is called in relation to the deno cache command proved to be challenging, but I noticed the warning when using the old private ENV var DENO_NPM_REGISTRY. I deleted the lock file, switched the deno cache command to use the private one, and saw the warning defined in RealNpmRegistryApi::default_url() in the output. Then when I ran it a second time, I got the above error output with no sign of the ENV var warning, which seems to indicate that the registry override is not being retrieved the same way on subsequent runs of deno cache when a lock file is present.

My full deno cache command (with a dummy Registry URL) that I committed to a Task command is:

$ NPM_CONFIG_REGISTRY=https://registry.mycompany.com/path/to/destination/ deno cache --reload --lock-write --node-modules-dir app.ts

[bartlomieju]

Thanks for the report. It's a bug in our lockfile implementation - we should store the registry URL in the lockfile.

[webbower]

That's what I suspected was missing.

Would it be worth also implementing that in the absence of the registry in the lockfile that it would use ENV override? For cases like mine, ALL NPM installations need to go through the private company Artifactory to be cached so the ENV override should always be present (unless Deno ever supported setting that in something akin to an .npmrc) so maybe the subsequent lookups should give the ENV override precedence over the registry setting in the lockfile?

[jimisaacs]

+1 to the point webbower just made. To be clear, the company that requires our company to do this, was NPM/Github/Microsoft. We hit the registry so much, we have been flagged to require all of our enterprise use cases to go through our private registry.

[jimisaacs]

I think we'd probably b fine if it just always used the ENV too. The only reason I can think of the registry in the lock file instead or in addition to this, is to support different registries per namespaces, which .npmrc supports. If there's no intention to support this, then I don't think the registry needs to be in the lock file. To be more clear, I think this particular bug could be fixed by always reading the correct ENV. Then when considering the registry per namespace use case, making a new lock file format could be floated.

[jimisaacs]

You know not even the registry per namespace would matter, because you still need to configure that outside of the lock file too. So really, the only reason would be to hydrate your cache strictly and only from an existing lock file. Therefore not requiring any further configuration or ENV settings.

[bartlomieju]

That's what I suspected was missing.

Would it be worth also implementing that in the absence of the registry in the lockfile that it would use ENV override? For cases like mine, ALL NPM installations need to go through the private company Artifactory to be cached so the ENV override should always be present (unless Deno ever supported setting that in something akin to an .npmrc) so maybe the subsequent lookups should give the ENV override precedence over the registry setting in the lockfile?

@dsherret what do you think about that?

[felipemullen]

I have a similar problem. I am using windmill.dev for scripting, which uses Deno for typescript executions. After a long struggle I am able to use private packages in our enterprise artifactory server, but I am unable to cache packages for in-browser script editing.

[AudriusButkevicius]

After a long struggle I am able to use private packages in our enterprise artifactory server

Can you elaborate how you did that?

[felipemullen]

@AudriusButkevicius I used a Verdaccio server with credentials to my private artifactory server. Verdaccio runs with docker compose, with a verdaccio.conf that has this:

# https://verdaccio.org/docs/configuration#uplinks
# a list of other known repositories we can talk to
uplinks:
  npmjs:
    cache: true
    url: https://registry.npmjs.org/

  artifactory:
    cache: false
    url: https://private.artifactory.myorg.com/api/npm/private-npm-repo/
    headers:
      Authorization: bearer abc123<><><>==

packages:
  '@myorg/*':
    access: $all
    publish: $authenticated
    unpublish: $authenticated
    proxy: artifactory

Once the package server is running, you can set NPM_CONFIG_REGISTRY to point to that server location, as described in the docs