search

aws / aws-sdk-js-v3

Modularized AWS SDK for JavaScript.
Apache License 2.0
3.04k stars 569 forks source link

Yarn PnP failures related to peer dependencies around client-sts #6258

Open ankon opened 2 months ago

ankon commented 2 months ago

Checkboxes for prior research

Describe the bug

We've been using Yarn's PnP linking to speed up builds and generally try to enforce strict dependencies.

Recently we noticed that builds using esbuild failed when not marking the @aws-sdk packages as external:

$ yarn
➤ YN0000: ┌ Resolution step
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.590.0 doesn't provide @aws-sdk/client-sts (p20ccd), requested by @aws-sdk/credential-provider-web-identity
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.590.0 doesn't provide @aws-sdk/client-sts (p5d9f9), requested by @aws-sdk/credential-provider-ini
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @aws-sdk/client-sts (p0057b), requested by @aws-sdk/credential-provider-web-identity
➤ YN0002: │ @aws-sdk/credential-provider-node@npm:3.609.0 doesn't provide @aws-sdk/client-sts (pe8c2b), requested by @aws-sdk/credential-provider-ini
➤ YN0002: │ @aws-sdk/credential-provider-sso@npm:3.590.0 doesn't provide @aws-sdk/client-sso-oidc (p6168f), requested by @aws-sdk/token-providers
➤ YN0002: │ @aws-sdk/credential-provider-sso@npm:3.609.0 doesn't provide @aws-sdk/client-sso-oidc (p7c0eb), requested by @aws-sdk/token-providers
➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0000: └ Completed
➤ YN0000: Done with warnings in 0s 550ms

Building with AWS SDK v3 as external:

$ esbuild index.js --target=es2020 --bundle --platform=node --format=cjs --minify --outfile=build/index.js --external:@aws-sdk/*

  build/index.js  246.6kb

⚡ Done in 55ms

Building with AWS SDK v3 bundled into the result:

$ esbuild index.js --target=es2020 --bundle --platform=node --format=cjs --minify --outfile=build/index.js
✘ [ERROR] Could not resolve "@aws-sdk/client-sts"

    ../../.yarn/__virtual__/@aws-sdk-credential-provider-web-identity-virtual-ff1c7036c6/0/cache/@aws-sdk-credential-provider-web-identity-npm-3.609.0-a2a0bf35c3-7a95a6c479.zip/node_modules/@aws-sdk/credential-provider-web-identity/dist-cjs/fromWebToken.js:32:113:
      32 │         const { getDefaultRoleAssumerWithWebIdentity } = await Promise.resolve().then(() => __importStar(require("@aws-sdk/client-sts")));
         ╵                                                                                                                  ~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on "@aws-sdk/client-sts",
  but the package "@aws-sdk/client-sts" has not been installed:

    ../../.pnp.cjs:1998:36:
      1998 │             ["@aws-sdk/client-sts", null],\
           ╵                                     ~~~~

  You can mark the path "@aws-sdk/client-sts" as external to exclude it from the bundle, which will
  remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

✘ [ERROR] Could not resolve "@aws-sdk/client-sts"

    ../../.yarn/__virtual__/@aws-sdk-credential-provider-ini-virtual-3b149592d0/0/cache/@aws-sdk-credential-provider-ini-npm-3.609.0-ce49945e2f-c8a2e83ff8.zip/node_modules/@aws-sdk/credential-provider-ini/dist-cjs/index.js:104:89:
      104 │     const { getDefaultRoleAssumer } = await Promise.resolve().then(() => __toESM(require("@aws-sdk/client-sts")));
          ╵                                                                                          ~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on "@aws-sdk/client-sts",
  but the package "@aws-sdk/client-sts" has not been installed:

    ../../.pnp.cjs:1684:36:
      1684 │             ["@aws-sdk/client-sts", null],\
           ╵                                     ~~~~

  You can mark the path "@aws-sdk/client-sts" as external to exclude it from the bundle, which will
  remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

✘ [ERROR] Could not resolve "@aws-sdk/client-sso-oidc"

    ../../.yarn/__virtual__/@aws-sdk-token-providers-virtual-0203582f7b/0/cache/@aws-sdk-token-providers-npm-3.609.0-044ab44290-e9cff1b949.zip/node_modules/@aws-sdk/token-providers/dist-cjs/index.js:51:79:
      51 │   const { SSOOIDCClient } = await Promise.resolve().then(() => __toESM(require("@aws-sdk/client-sso-oidc")));
         ╵                                                                                ~~~~~~~~~~~~~~~~~~~~~~~~~~

  The Yarn Plug'n'Play manifest says this package has a peer dependency on
  "@aws-sdk/client-sso-oidc", but the package "@aws-sdk/client-sso-oidc" has not been installed:

    ../../.pnp.cjs:2986:41:
      2986 │             ["@aws-sdk/client-sso-oidc", null],\
           ╵                                          ~~~~

  You can mark the path "@aws-sdk/client-sso-oidc" as external to exclude it from the bundle, which
  will remove this error and leave the unresolved path in the bundle. You can also surround this
  "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.

3 errors

SDK version number

@aws-sdk/client-sts@3.609.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.11.1

Reproduction Steps

https://github.com/ankon/20240705-aws-sdk-v3-peer-deps:

Run yarn to reproduce peer dependency warnings; then yarn build to build using bundling or yarn build-external to build with the AWS SDK marked as external.

Observed Behavior

See description: peer-dependency warnings, build failures.

Expected Behavior

Possible Solution

No response

Additional Information/Context

This seems to come up a lot of times, our quick search found:

richa-misra commented 1 month ago

Having the same issue with latest version aws-sdk version 3.624.0. Need to upgrade to use latest as it has change for fast-xml-parser but then fails on peer dependencies. Any options please?

wojtekmaj commented 1 month ago

If anything, you could use yarnrc.yml and packageExtensions to patch up incorrectly defined dependencies.

wojtekmaj commented 1 month ago

Here's are my error messages, a bit more detailed:

Package @aws-sdk/credential-provider-node@npm:3.622.0 is requested to provide @aws-sdk/client-sts by its descendants

@aws-sdk/credential-provider-node@npm:3.622.0
├─ @aws-sdk/credential-provider-ini@npm:3.622.0 [a1277] (via ^3.622.0)
│  └─ @aws-sdk/credential-provider-web-identity@npm:3.621.0 [a1277] (via ^3.621.0)
└─ @aws-sdk/credential-provider-web-identity@npm:3.621.0 [a1277] (via ^3.621.0)

✘ Package @aws-sdk/credential-provider-node@npm:3.622.0 does not provide @aws-sdk/client-sts.
Package @aws-sdk/credential-provider-sso@npm:3.622.0 is requested to provide @aws-sdk/client-sso-oidc by its descendants

@aws-sdk/credential-provider-sso@npm:3.622.0
└─ @aws-sdk/token-providers@npm:3.614.0 [aa9ff] (via ^3.614.0)

✘ Package @aws-sdk/credential-provider-sso@npm:3.622.0 does not provide @aws-sdk/client-sso-oidc.
Luke-lsm commented 1 week ago

As @wojtekmaj suggested, adding the dependencies in .yarnrc.yml will solve the issue temporarily while we wait for a fix, you can keep working through the missing dependency errors and adding them. For example my packageExtension is as follows: 

nodeLinker: pnp

yarnPath: .yarn/releases/yarn-4.4.1.cjs

packageExtensions:
  "@aws-sdk/credential-provider-node@>=3.609.0":
    dependencies:
      "@aws-sdk/client-sts": "*"
  "@aws-sdk/credential-provider-sso@>=3.609.0":
    dependencies:
      "@aws-sdk/client-sso-oidc": "*"