Yarn 2 (Yarn PnP) is not supported

lionskape commented 3 years ago

Describe the bug Yarn 2 (Yarn PNP) is not supported

To Reproduce Install app using yarn pnp. Try to configure and use amplify

Error text:

Module not found: @aws-amplify/api tried to access @aws-amplify/cache, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

Expected behavior Dependencies declared directly in packages, so yarn PnP requirements meet.

What is Configured? If applicable, please provide what is configured for Amplify CLI:

Which resources do you have configured? Auth
- If applicable, please provide your manual configuration example:
```
{
Auth: {
identityPoolId: 'XX-XXXX-X:XXXXXXXX-XXXX-1234-abcd-1234567890ab',
region: 'XX-XXXX-X',
}
}
```

Environment

``` npx envinfo --system --binaries --browsers --npmPackages --npmGlobalPackages ``` System: OS: macOS 11.2.1 CPU: (4) x64 Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz Memory: 54.26 MB / 16.00 GB Shell: 5.8 - /bin/zsh Binaries: Node: 12.20.1 - ~/.nvm/versions/node/v12.20.1/bin/node Yarn: 2.4.0 - ~/.nvm/versions/node/v12.20.1/bin/yarn npm: 6.14.10 - ~/.nvm/versions/node/v12.20.1/bin/npm Browsers: Chrome: 88.0.4324.182 Edge: 88.0.705.74 Firefox: 85.0 Safari: 14.0.3 npmGlobalPackages: npm: 6.14.10 yarn: 1.22.10

lionskape commented 3 years ago

As a temporal fix I used such yarn packageExtensions (auth package only. It is not a fix for all packages):

packageExtensions:
  '@aws-amplify/ui-react@*':
    dependencies:
      '@aws-amplify/core': '*'
  '@aws-amplify/ui-components@*':
    peerDependencies:
      '@aws-amplify/core': '*'
    dependencies:
      '@aws-amplify/storage': '*'
      '@aws-amplify/interactions': '*'
  '@aws-amplify/datastore@*':
    peerDependencies:
      '@aws-amplify/cache': '*'
    dependencies:
      'crypto-js': '*'
  '@aws-amplify/storage@*':
    dependencies:
      '@aws-sdk/protocol-http': '*'
      '@aws-sdk/fetch-http-handler': '*'
      '@aws-sdk/querystring-builder': '*'
  '@aws-amplify/api@*':
    peerDependencies:
      '@aws-amplify/core': '*'
      '@aws-amplify/cache': '*'

mariusbutuc commented 3 years ago

@lionskape thanks for surfacing this. I was also able to reproduce this as well while following the first chapter of @dabit3's Full Stack Serverless book + using Yarn 2 Modern. With your suggestion + the discussion between @atoiv & @milichev at https://github.com/aws-amplify/amplify-js/discussions/6048#discussioncomment-212237 and using the current versions of

"dependencies": {
  "@aws-amplify/ui-react": "^1.0.7",
  "aws-amplify": "^3.3.27",

I kept track of the "ambiguous and unsound" require calls, one by one, to compile this list in .yarnrc.yml:

# Failed to compile.
#
# Module not found: @aws-amplify/X tried to access @aws-amplify/Y, but it isn't declared in its dependencies;
# this makes the require call ambiguous and unsound.
packageExtensions:
  "@aws-amplify/api@*":
    dependencies:
      # ./.yarn/cache/@aws-amplify-api-npm-3.2.30-cfc9d17a70-ef156f3d1b.zip/node_modules/@aws-amplify/api/lib-esm/API.js
      "@aws-amplify/auth": "*"
      # idem
      "@aws-amplify/cache": "*"
      # idem
      "@aws-amplify/core": "*"
  "@aws-amplify/datastore@*":
    dependencies:
      # ./.yarn/cache/@aws-amplify-datastore-npm-2.9.16-25eb2e5416-04057925fb.zip/node_modules/@aws-amplify/datastore/lib-esm/sync/processors/subscription.js
      "@aws-amplify/auth": "*"
      # idem
      "@aws-amplify/cache": "*"
      # ./.yarn/cache/@aws-amplify-datastore-npm-2.9.16-25eb2e5416-04057925fb.zip/node_modules/@aws-amplify/datastore/lib-esm/util.js
      "crypto-js": "*"
  "@aws-amplify/storage@*":
    dependencies:
      # ./.yarn/cache/@aws-amplify-storage-npm-3.4.0-d253a241e3-d7248e6ddb.zip/node_modules/@aws-amplify/storage/lib-esm/providers/AWSS3ProviderManagedUpload.js
      "@aws-sdk/fetch-http-handler": "*"
      # ./.yarn/cache/@aws-amplify-storage-npm-3.4.0-d253a241e3-d7248e6ddb.zip/node_modules/@aws-amplify/storage/lib-esm/providers/axios-http-handler.js
      "@aws-sdk/protocol-http": "*"
      # idem
      "@aws-sdk/querystring-builder": "*"
  "@aws-amplify/ui-components@*":
    dependencies:
      # ./.yarn/$$virtual/@aws-amplify-ui-components-virtual-b697fad612/0/cache/@aws-amplify-ui-components-npm-1.2.0-93739495ef-1e8458e916.zip/node_modules/@aws-amplify/ui-components/dist/esm-es5/Translations-c833f663.js
      "@aws-amplify/auth": "*"
      # ./.yarn/$$virtual/@aws-amplify-ui-components-virtual-b697fad612/0/cache/@aws-amplify-ui-components-npm-1.2.0-93739495ef-1e8458e916.zip/node_modules/@aws-amplify/ui-components/dist/esm-es5/amplify-amazon-button_5.entry.js
      "@aws-amplify/core": "*"
      # ./.yarn/$$virtual/@aws-amplify-ui-components-virtual-b697fad612/0/cache/@aws-amplify-ui-components-npm-1.2.0-93739495ef-1e8458e916.zip/node_modules/@aws-amplify/ui-components/dist/esm-es5/amplify-chatbot.entry.js
      "@aws-amplify/interactions": "*"
      # ./.yarn/$$virtual/@aws-amplify-ui-components-virtual-b697fad612/0/cache/@aws-amplify-ui-components-npm-1.2.0-93739495ef-1e8458e916.zip/node_modules/@aws-amplify/ui-components/dist/esm-es5/amplify-s3-album.entry.js
      "@aws-amplify/storage": "*"
  "@aws-amplify/ui-react@*":
    dependencies:
      # ./.yarn/$$virtual/@aws-amplify-ui-react-virtual-3dde9c981e/0/cache/@aws-amplify-ui-react-npm-1.0.7-b3880dea19-346276e124.zip/node_modules/@aws-amplify/ui-react/lib-esm/withAuthenticator.js
      "@aws-amplify/auth": "*"
      # idem
      "@aws-amplify/core": "*"

This allowed the trivial CRA + amplify + auth app to run successfully, yet left me wondering if this the right way to handle this incompatibility.

Environment

```sh yarn dlx envinfo --system --binaries --browsers --npmPackages --npmGlobalPackages ``` ``` System: OS: macOS 11.3 CPU: (8) x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz Memory: 218.75 MB / 16.00 GB Shell: 5.8 - /bin/zsh Binaries: Node: 12.21.0 - /private/var/folders/r0/ygyqhzmn7tv_gln5hhf7g5r00000gp/T/xfs-7219e856/node Yarn: 2.4.1 - /private/var/folders/r0/ygyqhzmn7tv_gln5hhf7g5r00000gp/T/xfs-7219e856/yarn npm: 6.14.11 - ~/.asdf/installs/nodejs/12.21.0/bin/npm Watchman: 4.9.0 - /usr/local/bin/watchman Browsers: Chrome: 90.0.4430.93 Firefox: 67.0 Safari: 14.1 npmGlobalPackages: nodemon: 2.0.7 npm: 7.11.1 ```

pfried commented 1 year ago

This is a larger issue with the way the package.json files are setup. My guess is that all packages except aws-amplify should declare @aws-amplify/core (and possibly others) as a peer dependency.

Three ways to use the library:

Monolothic aws-amplify -> In this case everything should work correctly as it is
Modular import relying on hoisting to the top -> Works because e.g. @aws-ampify/core gets hoisted and all packages requiring it will get the same dependency even if you install @aws-amplify/pubsub (if all the versions match, which should be the case because I guess the numbers are created by a CI process, but this is still prone for user errors where someone would install a non matching lib with the same symptoms as 3.)
Modular library import without hoisting -> This currently does not work correctly (e.g. yarn 2 with nmHoistingLimits). Say you want to import @aws-amplify/pubsub then this package does not work without the user installing @aws-amplify/core anyways (you will need to call configure). In a typical installation you will have multiple packages (say api+pubsub) all declaring @aws-amplify/core as a dependency which will all have their own version in their node_modules. This leads to a broken build as e.g. Hub will stop working when each packages includes their own version of it

If the they were a peer depdency cases 2. and 3. would work correctly. You get a warning for 2. and 3. would then result in the same @aws-amplify/core pacakge shared by all dependents.

Sure enough this will require to have @aws-amplify/core installed in package.json but that would be fine. If we see all the other packages as plugins to core it makes sense

aws-amplify / amplify-js

Yarn 2 (Yarn PnP) is not supported #7772