[Bug?]: No migration troubleshooting docs for error `... isn't supported by any available resolver`

nettybun commented 1 year ago

Self-service

[X] I'd be willing to implement a fix

Describe the bug

Hi I'm trying to update our repo from Yarn 1.22 to Yarn 3 and was following the migration guide: https://yarnpkg.com/getting-started/migration

This is the error I'm getting:

➤ YN0001: │ Error: @splunk/dashboard-utils@npm:25.1.0: @splunk/dashboard-utils@npm:25.1.0::__archiveUrl=https%3A%2F%2Frepo.splunk.com%2Fartifactory%2Fapi%2Fnpm%2Fnpm%2F%40splunk%2Fdashboard-utils%2F-%2F%40splunk%2Fdashboard-utils-25.1.0.tgz%23 isn't supported by any available resolver
    at Wf.getResolverByDescriptor (/repo/.yarn/releases/yarn-3.3.0.cjs:391:1647)
    at Wf.getCandidates (/repo/.yarn/releases/yarn-3.3.0.cjs:391:1235)
    at fQ.getCandidates (/repo/.yarn/releases/yarn-3.3.0.cjs:435:3324)
    at Wf.getCandidates (/repo/.yarn/releases/yarn-3.3.0.cjs:391:1264)
    at /repo/.yarn/releases/yarn-3.3.0.cjs:439:7695
    at Rf (/repo/.yarn/releases/yarn-3.3.0.cjs:390:8965)
    at ge (/repo/.yarn/releases/yarn-3.3.0.cjs:439:7675)
➤ YN0000: └ Completed in 2m 11s
➤ YN0000: Failed with errors in 2m 11s

I checked the https://yarnpkg.com/getting-started/migration#troubleshooting and searched GH issues for isn't supported by any available resolver but was unable to an existing issue.

Maybe https://github.com/yarnpkg/berry/blob/master/packages/gatsby/content/getting-started/migration.md could be updated with docs on how to debug/proceed when this happens?

This is a bug with documentation, since I see your issue template says Note that bugs without minimal reproductions will be closed as non-actionable. and I don't have a minimal repro case for the failed install; that's the problem. This is a failed migration of a private monorepo with 15 packages and 20000 lines of yarn.lock - I don't know where to start troubleshooting to be able to create a minimal repro case, and I think the docs could help with that. I understand that ideally you don't want users to need to do this, but it will be necessary to some people - there are a dozen teams at my company all using yarn-classic v1 because they've run until similar errors and given up. I think have good troubleshooting docs could help.

I'm willing to submit a docs PR if I can gather some troubleshooting steps that would help others. Right now my best guess is "Clone the yarn berry repo following CONTRIBUTING.MD, run yarn build:cli --no-minify, ...somehow attach that to a Node Debugger, set breakpoints, re-run the erroring yarn install again". Is this the right track? Thank you!

To reproduce

Follow the migration guide for a complex monorepo

Use a "node_modules"-based .yarnrc.yml:

yarnPath: .yarn/releases/yarn-3.3.0.cjs
npmRegistryServer: "https://repo.splunk.com/artifactory/api/npm/npm"
nodeLinker: node-modules

Fail the migration
Check troubleshooting guide, GH issues, etc...
Reluctantly open an issue 😅

Environment

System:
    OS: macOS 12.6
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
  Binaries:
    Node: 16.14.0 - /private/var/folders/qv/pc1l_8w90b18nk2f5ljqf4hh0000gp/T/xfs-50824e24/node
    Yarn: 3.3.0 - /private/var/folders/qv/pc1l_8w90b18nk2f5ljqf4hh0000gp/T/xfs-50824e24/yarn
    npm: 8.3.1 - /usr/local/bin/npm
  npmPackages:
    jest: ^26.0.0 => 26.6.3

Additional context

No response

florianeichin commented 1 year ago

This would be awesome. I also have this issue by upgrading to Yarn 3.3.0 While running the yarnpkg/doctor tool I get this type of lines for all dependencies in my tree:

➤ YN0000: ┌ /root/code/my-tool/package.json ➤ YN0000: │ Resolving @types/adm-zip@~0.4.32 errored with @types/adm-zip@~0.4.32 isn't supported by any available resolver ➤ YN0000: │ Resolving @types/buffer-from@^1.1.0 errored with @types/buffer-from@^1.1.0 isn't supported by any available resolver ➤ YN0000: │ Resolving @types/glob@~7.1.1 errored with @types/glob@~7.1.1 isn't supported by any available resolver ➤ YN0000: │ Resolving @xmldom/xmldom@~0.7.5 errored with @xmldom/xmldom@~0.7.5 isn't supported by any available resolver

The only resolve version in my yarn.lock is 1.22.1 so 1.9+

But I can run yarn install successfully with PnP... somehow looks strange to me

nelsonpecora commented 1 year ago

I'm having the exact same issue upgrading from yarn v1 to v3 using a private registry, and I think it has to do with the # at the end of the __archiveUrl.

https%3A%2F%2Frepo.splunk.com%2Fartifactory%2Fapi%2Fnpm%2Fnpm%2F%40splunk%2Fdashboard-utils%2F-%2F%40splunk%2Fdashboard-utils-25.1.0.tgz%23 urldecodes to https://repo.splunk.com/artifactory/api/npm/npm/@splunk/dashboard-utils/-/@splunk/dashboard-utils-25.1.0.tgz#

I know that yarn requires these kinds of urls to have .tgz (or .tar.gz) at the end, and I suspect something internally is seeing .tgz# and failing. Unfortunately, I'm not sure if this is an issue with yarn itself or with the registry. It looks like we both use artifactory, which in yarn v1 would give resolved urls (in yarn.lock) looking like this:

https://packages.company.com/api/npm/registry-name/@scope/package-name/-/@scope/package-name-1.2.3.tgz#1de9a05f927192e7f03cf4c5b5f534e9bbb1f836

I found an open issue against artifactory that mentioned how their url schema is somewhat nonstandard, but that issue doesn't mention the hash at the end. I'm not 100% sure what that hash is, or if it's returned to yarn's resolver, but I suspect it might be. Yarn's resolver might be attempting to parse it, but ending up with .tgz#.

Would anybody on the yarn team have more insight into this?

nelsonpecora commented 1 year ago

Good news and bad news: I think this issue pops up specifically when upgrading yarn v1 to v3 in a repo that has a v1-style yarn.lock that references artifactory urls. If I delete my yarn.lock and re-run yarn install, it works and generates a lockfile without the hashes, e.g.

"@scope/package-name@npm:1.2.3":
  version: 1.2.3
  resolution: "@scope/package-name@npm:1.2.3::__archiveUrl=https%3A%2F%2Fpackages.company.com%2Fapi%2Fnpm%2Fregistry-name%2F%40scope%2Fpackage-name%2F-%2F%40scope%2Fpackage-name-1.2.3.tgz"
  dependencies:
    "@scope/other-package-name": 1.0.1
  checksum: <hash>
  languageName: node
  linkType: hard

Urldecoding that new __archiveUrl gives me:

https://packages.company.com/api/npm/registry-name/@scope/package-name/-/@scope/package-name-1.2.3.tgz

I'm not sure if there's an easy way to solve this without blowing away the lockfile, though. I tried removing the hashes from the end of each url in the old lockfile, but that gave me the same error as before.

Update: Solution

I may have gotten it working by replacing all of those urls in the v1 yarn.lock with two regexes (replace packages.company.com with your own artifactory host):

For scoped packages:

resolved "https://packages\.company\.com/api/npm/(.*?)/@(.*?)/(.*?)/-/(?:.*)-(\d+\.\d+\.\d+.*?)\.tgz#(?:.*?)"

For replacements:

registry is $1 scope is $2 package is $3 version is $4

resolved "@$2/$3@npm:$4::__archiveUrl=https%3A%2F%2Fpackages.company.com%2Fapi%2Fnpm%2F$1%2F%40$2%2F$3%2F-%2F%40$2%2F$3-$4.tgz"

For unscoped packages:

resolved "https://packages\.company\.com/api/npm/(.*?)/(.*?)/-/(?:.*)-(\d+\.\d+\.\d+.*?)\.tgz#(?:.*?)"

For replacements:

registry is $1 package is $2 version is $3

resolved "$2@npm:$3::__archiveUrl=https%3A%2F%2Fpackages.company.com%2Fapi%2Fnpm%2F$1%2F$2%2F-%2F$2-$3.tgz"

Once I make those replacements in yarn.lock, I re-ran yarn install and it generated a proper v3-style lockfile!

mcenkar commented 1 year ago

Hi @heyheyhello @florianeichin @nelsonpecora,

I think I have same/similar issue in #5156,

Do you still have repo where you could try my fix and see if it fixes this problem? Either apply patch from #5156 and build yarn or replace yarn-X.Y.Z.js with attached yarn.js.zip

nettybun commented 1 year ago

This issue would be fixed by #5212 but note that #5156 is still an open bug - my ticket here is just about the lack of docs/help for debugging.

nuykon commented 5 months ago

Hi @heyheyhello @florianeichin @nelsonpecora,

I think I have same/similar issue in #5156,

Do you still have repo where you could try my fix and see if it fixes this problem? Either apply patch from #5156 and build yarn or replace yarn-X.Y.Z.js with attached yarn.js.zip

I have updated yarn from v1 to version 4.1.1 and I have the same error when trying to install a package from gitlab registry(https://mydomain.ru/api/v4/projects/97/packages/npm/@myscope/ui/-/myscope/ui-2.8.8.tgz#218d812a52e0dda867846812c7bc628b016dd4ba)

Error: @myscope/ui@npm:^2.8.8: @myscope/ui@npm:2.8.8::__archiveUrl=https%3A%2F%2Fmydomain.ru%2Fapi%2Fv4%2Fprojects%2F97%2Fpackages%2Fnpm%2F%40myscope%2Fui%2F-%2F%40myscope%2Fui-2.8.8.tgz%23 isn't supported by any available resolver

After applying the patch from @mcenkar comment, the installation worked, but any other commands give an error.

yarn dev Internal Error: @myscope/hm-portal@workspace:.: This package doesn't seem to be present in your lockfile; run "yarn install" to update the lockfile

yarnpkg / berry