Yarn PnP failures related to peer dependencies around client-sts

ankon commented 4 months ago

Describe the bug

We've been using Yarn's PnP linking to speed up builds and generally try to enforce strict dependencies.

Recently we noticed that builds using esbuild failed when not marking the @aws-sdk packages as external:

$ yarn
➤ YN0000: ┌ Resolution step
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.590.0 doesn't provide @aws-sdk/client-sts (p20ccd), requested by @aws-sdk/credential-provider-web-identity
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.590.0 doesn't provide @aws-sdk/client-sts (p5d9f9), requested by @aws-sdk/credential-provider-ini
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @aws-sdk/client-sts (p0057b), requested by @aws-sdk/credential-provider-web-identity
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @aws-sdk/client-sts (pe8c2b), requested by @aws-sdk/credential-provider-ini
➤ YN0002: │ @aws-sdk/credential-provider-sso@npm:3.590.0 doesn't provide @aws-sdk/client-sso-oidc (p6168f), requested by @aws-sdk/token-providers
➤ YN0002: │ @aws-sdk/credential-provider-sso@npm:3.609.0 doesn't provide @aws-sdk/client-sso-oidc (p7c0eb), requested by @aws-sdk/token-providers
➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0000: └ Completed
➤ YN0000: Done with warnings in 0s 550ms

Building with AWS SDK v3 as external:

$ esbuild index.js --target=es2020 --bundle --platform=node --format=cjs --minify --outfile=build/index.js --external:@aws-sdk/*

  build/index.js  246.6kb

⚡ Done in 55ms

Building with AWS SDK v3 bundled into the result:

$ esbuild index.js --target=es2020 --bundle --platform=node --format=cjs --minify --outfile=build/index.js
✘ [ERROR] Could not resolve "@aws-sdk/client-sts"

    ../../.yarn/__virtual__/@aws-sdk-credential-provider-web-identity-virtual-ff1c7036c6/0/cache/@aws-sdk-credential-provider-web-identity-npm-3.609.0-a2a0bf35c3-7a95a6c479.zip/node_modules/@aws-sdk/credential-provider-web-identity/dist-cjs/fromWebToken.js:32:113:
      32 │         const { getDefaultRoleAssumerWithWebIdentity } = await Promise.resolve().then(() => __importStar(require("@aws-sdk/client-sts")));
         ╵                                                                                                                  ~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on "@aws-sdk/client-sts",
  but the package "@aws-sdk/client-sts" has not been installed:

    ../../.pnp.cjs:1998:36:
      1998 │             ["@aws-sdk/client-sts", null],\
           ╵                                     ~~~~

  You can mark the path "@aws-sdk/client-sts" as external to exclude it from the bundle, which will
  remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

✘ [ERROR] Could not resolve "@aws-sdk/client-sts"

    ../../.yarn/__virtual__/@aws-sdk-credential-provider-ini-virtual-3b149592d0/0/cache/@aws-sdk-credential-provider-ini-npm-3.609.0-ce49945e2f-c8a2e83ff8.zip/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/index.js:104:89:
      104 │     const { getDefaultRoleAssumer } = await Promise.resolve().then(() => __toESM(require("@aws-sdk/client-sts")));
          ╵                                                                                          ~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on "@aws-sdk/client-sts",
  but the package "@aws-sdk/client-sts" has not been installed:

    ../../.pnp.cjs:1684:36:
      1684 │             ["@aws-sdk/client-sts", null],\
           ╵                                     ~~~~

  You can mark the path "@aws-sdk/client-sts" as external to exclude it from the bundle, which will
  remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

✘ [ERROR] Could not resolve "@aws-sdk/client-sso-oidc"

    ../../.yarn/__virtual__/@aws-sdk-token-providers-virtual-0203582f7b/0/cache/@aws-sdk-token-providers-npm-3.609.0-044ab44290-e9cff1b949.zip/node_modules/@aws-sdk/token-providers/dist-cjs/index.js:51:79:
      51 │   const { SSOOIDCClient } = await Promise.resolve().then(() => __toESM(require("@aws-sdk/client-sso-oidc")));
         ╵                                                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on
  "@aws-sdk/client-sso-oidc", but the package "@aws-sdk/client-sso-oidc" has not been installed:

    ../../.pnp.cjs:2986:41:
      2986 │             ["@aws-sdk/client-sso-oidc", null],\
           ╵                                          ~~~~

  You can mark the path "@aws-sdk/client-sso-oidc" as external to exclude it from the bundle, which
  will remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

3 errors

Expected Behavior

yarn install properly installs without warnings.
Builds work in both --external:@aws-sdk/* and bundled cases.

Current Behavior

See description: peer-dependency warnings, build failures.

Reproduction Steps

https://github.com/ankon/20240705-aws-sdk-v3-peer-deps:

Run yarn to reproduce peer dependency warnings; then yarn build to build using bundling or yarn build-external to build with the AWS SDK marked as external.

Possible Solution

No response

Additional Information/Context

This seems to come up a lot of times, our quick search found:

SDK version used

^3.609.0

Environment details (OS name and version, etc.)

Fedora 40, yarn.

ankon commented 4 months ago

FWIW: The only sane thing we could come up with that would not require us to essentially maintain your peer dependency mess is to switch to the yarn node-modules linker, and take the performance hit on every build.

ankon commented 4 months ago

Closing, filed in the wrong repository: https://github.com/aws/aws-sdk-js-v3/issues/6258

aws / aws-sdk-js