[Bug?]: yarn 4.0.2 ignores npmAlwaysAuth in ${PROJECT}/.yarnrc.yml

[gustafc]

Self-service

• [x] I'd be willing to implement a fix

Describe the bug

(This is sort of a follow-up to #4341, where I tried to upgrade to yarn 3, which didn't work because it didn't use the token saved by yarn npm login.)

TL;DR: yarn install doesn't respect npmAlwaysAuth in the project-local .yarnrc.yml. It works if you log in using yarn npm login --always-auth.

Current situation

• Company has a Nexus server which both serves internal NPM packages and acts as a proxy to NPM public
• Nexus server requires auth for all operations
• Company's projects use yarn classic, configured with .npmrc to use the above Nexus registry:

  always-auth=true
  registry=https://nexus.example.com/repository/npm-all/

• We don't have any .yarnrc files in the projects; .npmrc is sufficient
• We do not use scopes or such
• All packages are fetched through the Nexus registry
• We'd like to migrate to yarn modern

The problem

• I followed the migration steps which means I converted my .npmrc to .yarnrc.yml thus:

  nodeLinker: node-modules
  npmAlwaysAuth: true
  npmRegistryServer: "https://nexus.example.com/repository/npm-all/"

• After converting the config to modern format, I did yarn npm login which logged me in to our Nexus server, and wrote the token to ~/.yarnrc.yml as expected
• Running yarn npm whoami tells me I'm logged in:

  yarn npm whoami
  ➤ YN0000: gustaf
  ➤ YN0000: Done in 0s 74ms

• However, running yarn install fails with following output:

  ➤ YN0000: ┌ Fetch step
  ➤ YN0041: │ accepts@npm:1.3.8: Invalid authentication (as an anonymous user)
  ➤ YN0041: │ acorn-loose@npm:8.4.0: Invalid authentication (as an anonymous user)
  *** snip
  ➤ YN0041: │ yargs@npm:15.4.1: Invalid authentication (as an anonymous user)
  ➤ YN0041: │ yauzl@npm:2.10.0: Invalid authentication (as an anonymous user)
  ➤ YN0000: └ Completed in 1s 250ms
  ➤ YN0000: · Failed with errors in 1s 359ms

  I get the same error if I try to yarn add a new package.

A bit of sleuthing (why can't I do yarn install --verbose?) has lead me to the following observations:

• If I change the npmRegistryServer to use a hostname which doesn't exist, yarn install fails with getaddrinfo ENOTFOUND, so it actually tries to fetch from my company's Nexus server
• Access logs for the Nexus server confirm the above conclusion, I see a bunch of unauthorized requests
• If I curl the URLs in the access logs using the token in ~/.yarnrc.yml it works fine, so both token and URLs are correct
• If I remove the token from ~/.yarnrc.yml, install fails with "No authentication configured for request", so it does actually seem to at least read the token

The only conclusion I can draw from this is that it doesn't use the token.

To reproduce

I couldn't figure out how to make Sherlock start a Nexus server for reproduction, so here's an old-fashioned shell script which

1. creates a new project in /tmp with a local Nexus configured,
2. starts a local Nexus using Docker,
3. logs in to the local Nexus server,
4. tries to yarn add a package, which triggers the error

If you change yarn npm login in the script to yarn npm login --always-auth, step 4 will work.

#!/bin/bash
set -exuo pipefail

UNIQUE_STRING=$(date '+%Y%m%d-%H%M%S')
PROJECT=/tmp/yarn4-auth-repro-$UNIQUE_STRING
# Step 1: Create a new project in /tmp
mkdir -p $PROJECT
cd $_
yarn init -2
echo '
unsafeHttpWhitelist:
  - localhost
npmRegistryServer: "http://localhost:8081/repository/npm-all"
npmAlwaysAuth: true
' >>.yarnrc.yml

# Step 2: Set up Nexus
NEXUS_CONTAINER=nexus-repro-container-$UNIQUE_STRING

docker run --rm --detach -p 8081:8081 --name $NEXUS_CONTAINER sonatype/nexus3@sha256:66fe12b1eb3e97bae72eb3c2c4e436499d41ff144cdfd1dcd0718df738304732
function cleanup {
  read -p "Stop Nexus docker container $NEXUS_CONTAINER? (y/n) " STOP_CONTAINER
  if [[ $STOP_CONTAINER == y* ]]; then
    docker stop $NEXUS_CONTAINER
  else
    echo "You can play around in the project by going to the following directory:"
    echo "  cd $PROJECT"
    echo "To stop the Nexus container, run:"
    echo "  docker stop $NEXUS_CONTAINER"
  fi
}
trap cleanup EXIT

while ! docker logs $NEXUS_CONTAINER | grep 'Started Sonatype Nexus OSS' ; do echo "Waiting for $NEXUS_CONTAINER to start..."; sleep 2; done
printf '\a'
NEXUS_USER=admin
NEXUS_PASS="$(docker exec $NEXUS_CONTAINER cat /nexus-data/admin.password)"
# # Change admin password
# curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/rest/internal/ui/onboarding/change-admin-password' -X PUT --data-raw 'admin123'
# sleep 1
# NEXUS_PASS=admin123
# Disallow anonymous access
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_AnonymousSettings","method":"update","data":[{"enabled":false,"userId":"anonymous","realmName":"NexusAuthorizingRealm"}],"type":"rpc","tid":14}'

# Create npm repo proxy
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_Repository","method":"create","data":[{"attributes":{"npm":{"removeNonCataloged":false,"removeQuarantinedVersions":false},"proxy":{"remoteUrl":"https://registry.npmjs.org","contentMaxAge":1440,"metadataMaxAge":1440},"httpclient":{"blocked":false,"autoBlock":true,"connection":{"useTrustStore":false}},"storage":{"blobStoreName":"default","strictContentTypeValidation":true},"negativeCache":{"enabled":true,"timeToLive":1440},"cleanup":{"policyName":[]}},"name":"npm-proxy","format":"","type":"","url":"","online":true,"routingRuleId":"","authEnabled":false,"httpRequestSettings":false,"recipe":"npm-proxy"}],"type":"rpc","tid":31}'

# Create npm repo group (which is my real setup)
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_Repository","method":"create","data":[{"attributes":{"storage":{"blobStoreName":"default","strictContentTypeValidation":true},"group":{"memberNames":["npm-proxy"]}},"name":"npm-all","format":"","type":"","url":"","online":true,"recipe":"npm-group"}],"type":"rpc","tid":44}'

# Make NPM login work <https://issues.sonatype.org/browse/NEXUS-20170>
curl --fail --verbose -u "$NEXUS_USER:$NEXUS_PASS" 'http://localhost:8081/service/extdirect' -X POST -H 'Content-Type: application/json' --data-raw '{"action":"coreui_RealmSettings","method":"update","data":[{"realms":["NexusAuthenticatingRealm","NexusAuthorizingRealm","NpmToken"]}],"type":"rpc","tid":43}'

# Step 3: Login
(echo "$NEXUS_USER"; sleep 2; echo "$NEXUS_PASS") | yarn npm login
yarn npm whoami

# Step 4: Add any random package to make sure auth works (this will fail)
yarn add mkdirp@latest

Environment

System:
    OS: macOS 14.2.1
    CPU: (14) arm64 Apple M3 Max
  Binaries:
    Node: 20.9.0 - /private/var/folders/3x/6k490qcx7hg9tvzfxy1k8tyc0000gn/T/xfs-d788f556/node
    Yarn: 4.0.2 - /private/var/folders/3x/6k490qcx7hg9tvzfxy1k8tyc0000gn/T/xfs-d788f556/yarn
    npm: 10.1.0 - ~/.nvm/versions/node/v20.9.0/bin/npm

Additional context

No response

[gustafc]

No response for over a week. Is there anything more I can do to help? I'm willing to provide a fix it if someone points me in the right direction, I'm not familiar with the yarn codebase.

[koshic]

@gustafc try yarn npm login --always-auth. As far as I remember, yarn has a problems with merging authorization settings comes from ~/.yarnrc.yml & project .yarnrc.yml: auth settings treated like a single block (per repo), so empty 'alwaysAuth' in global config + 'alwaysAuth=true' in local config will produce empty 'alwaysAuth' in effective config for the particular repo (see yarn config output).

[gustafc]

@koshic Thanks, adding --always-auth works 🌟 So I'll rename the bug so reflect that the problem is that npmAlwaysAuth in project config is ignored.