search

yarnpkg / berry

📦🐈 Active development trunk for Yarn ⚒
https://yarnpkg.com
BSD 2-Clause "Simplified" License
7.38k stars 1.11k forks source link

[Bug]: HTTP urls do not support query parameters and do not support extensionless URLs #5929

Closed nicolo-ribaudo closed 11 months ago

nicolo-ribaudo commented 11 months ago

Self-service

Describe the bug

I am trying to add this tgz as a dependency: https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a

However, Yarn throws an error:

➤ YN0000: ┌ Resolution step
➤ YN0001: │ Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn?[...] isn't supported by any available resolver
    at kf.getResolverByDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1647)
    at kf.bindDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1036)
    at reduceDependency (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:28690)
    at Ha.reduceHook (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:392:3512)
    at _ (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:439:7045)
    at async Promise.allSettled (index 15)
    at async io (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:10398)
➤ YN0000: └ Completed
➤ YN0000: Failed with errors in 0s 135ms

I also tried to remove the query parameter, and it still throws:

➤ YN0000: ┌ Resolution step
➤ YN0001: │ Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn isn't supported by any available resolver
    at kf.getResolverByDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1647)
    at kf.bindDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1036)
    at reduceDependency (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:28690)
    at Ha.reduceHook (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:392:3512)
    at _ (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:439:7045)
    at async Promise.allSettled (index 15)
    at async io (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:10398)
➤ YN0000: └ Completed
➤ YN0000: Failed with errors in 0s 142ms

For the case without the query parameter, I can add #.tgz at the end of the URL. This is meaningless (it's just a URL hash that the server ignores), but it makes Yarn happy: https://gitpkg.now.sh/acornjs/acorn/acorn#.tgz

However, this hack does not work when there is a query parameter (https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a#.tgz).

To reproduce

await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a",
  },
})).resolves.not.toThrow();
await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn",
  },
})).resolves.not.toThrow();
await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn#.tgz",
  },
}));
await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a#.tgz",
  },
})).resolves.not.toThrow();

Environment

System:
    OS: macOS 14.0
    CPU: (8) arm64 Apple M2
  Binaries:
    Node: 20.8.0 - /private/var/folders/bg/d5r9tspx5hn87zp52mcl82dh0000gn/T/xfs-1a04181c/node
    Yarn: 3.6.2 - /private/var/folders/bg/d5r9tspx5hn87zp52mcl82dh0000gn/T/xfs-1a04181c/yarn
    npm: 10.1.0 - /usr/local/bin/npm
    bun: 1.0.0 - ~/.bun/bin/bun
  npmPackages:
    jest: ^29.6.2 => 29.6.2 

We couldn't upgrade to Yarn 4 yet, apologies if this has been already fixed :sweat_smile:

Additional context

Regardless of the query parameter, URLs extensions are meaningless. I could serve a JS file at http://example.com/foo.tar, or a tarball at http://example.com/foo.js. On the web, the file type is described by the mime type in the HTTP response and not by the "extension".

yarnbot commented 11 months ago

We couldn't reproduce your issue (all the assertions passed on master).

yarnbot commented 11 months ago

This issue reproduces on master:

Error: expect(received).resolves.not.toThrow()

Received promise rejected instead of resolved
Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js install

➤ YN0000: · Yarn 4.0.1-dev
➤ YN0000: ┌ Resolution step
::group::Resolution step
➤ YN0001: │ Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn?[...] isn't supported by any available resolver
    at MultiResolver.getResolverByDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:73:13)
    at MultiResolver.bindDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:31:27)
    at Project.preparePackage (/github/workspace/packages/yarnpkg-core/sources/Project.ts:737:30)
    at startPackageResolution (/github/workspace/packages/yarnpkg-core/sources/Project.ts:834:21)
    at async Promise.allSettled (index 0)
    at Object.allSettledSafe (/github/workspace/packages/yarnpkg-core/sources/miscUtils.ts:89:19)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:939:9)
    at StreamReport.startProgressPromise (/github/workspace/packages/yarnpkg-core/sources/Report.ts:167:14)
    at Project.resolveEverything (/github/workspace/packages/yarnpkg-core/sources/Project.ts:816:5)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:1775:7)
::endgroup::
➤ YN0001: Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn?[...] isn't supported by any available resolver
    at MultiResolver.getResolverByDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:73:13)
    at MultiResolver.bindDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:31:27)
    at Project.preparePackage (/github/workspace/packages/yarnpkg-core/sources/Project.ts:737:30)
    at startPackageResolution (/github/workspace/packages/yarnpkg-core/sources/Project.ts:834:21)
    at async Promise.allSettled (index 0)
    at Object.allSettledSafe (/github/workspace/packages/yarnpkg-core/sources/miscUtils.ts:89:19)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:939:9)
    at StreamReport.startProgressPromise (/github/workspace/packages/yarnpkg-core/sources/Report.ts:167:14)
    at Project.resolveEverything (/github/workspace/packages/yarnpkg-core/sources/Project.ts:816:5)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:1775:7)
➤ YN0000: └ Completed
➤ YN0000: · Failed with errors in 0s 114ms
]
    at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-9b3e8262b0.zip/node_modules/expect/build/index.js:138:15)
    at module.exports (evalmachine.<anonymous>:2:7)
    at /github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:57:19
    at executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:18:22)
    at executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:25:18)
    at ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:26:76)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-312910c71e.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-312910c71e.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-312910c71e.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)
arcanis commented 11 months ago

Duplicate of #2437, although this other thread is closed. We probably should improve the situation somehow.

arcanis commented 11 months ago

Will be fixed with https://github.com/yarnpkg/berry/pull/5931