yarn 2 / Plug N Play Support

[Jordan-Hall]

Please make sure you have read the submission guidelines before posting an issue

Prerequisites

Yarn version 2.0.0-rc.27

Expected Behavior

Should work correctly like before

Current Behavior

Build failure

Failure Information (for bugs)

An unhandled exception occurred: Cannot locate the 'node_modules' directory. See "AppData\Local\Temp\ng-mO3oFO\angular-errors.log" for further details.

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. yarn dlx @yarnpkg/doctor .
2. yarn install
3. yarn start

Context

Yarn has finally released a usable version two. If you using webpack 4 you need to enable PlugNPlay which is native to webpack 5

https://yarnpkg.com/advanced/migration https://yarnpkg.com/advanced/pnpapi

Failure Logs

[error] Error: Cannot locate the 'node_modules' directory.
    at NgccProcessor.findNodeModulesDirectory (C:\dev\lec-migration\.yarn\$$virtual\@ngtools-webpack-virtual-9bdca6ad48\0\cache\@ngtools-webpack-npm-9.0.0-rc.10-6bfaf96715-1.zip\node_modules\@ngtools\webpack\src\ngcc_processor.js:113:15)
    at new NgccProcessor (C:\dev\lec-migration\.yarn\$$virtual\@ngtools-webpack-virtual-9bdca6ad48\0\cache\@ngtools-webpack-npm-9.0.0-rc.10-6bfaf96715-1.zip\node_modules\@ngtools\webpack\src\ngcc_processor.js:31:43)
    at C:\dev\lec-migration\.yarn\$$virtual\@ngtools-webpack-virtual-9bdca6ad48\0\cache\@ngtools-webpack-npm-9.0.0-rc.10-6bfaf96715-1.zip\node_modules\@ngtools\webpack\src\angular_compiler_plugin.js:554:33
    at SyncHook.eval [as call] (eval at create (C:\dev\lec-migration\.yarn\cache\tapable-npm-1.1.3-f1c2843426-1.zip\node_modules\tapable\lib\HookCodeFactory.js:19:10), <anonymous>:7:1)
    at SyncHook.lazyCompileHook (C:\dev\lec-migration\.yarn\cache\tapable-npm-1.1.3-f1c2843426-1.zip\node_modules\tapable\lib\Hook.js:154:20)
    at Object.webpack [as webpackFactory] (C:\dev\lec-migration\.yarn\cache\webpack-npm-4.41.2-efbe0580aa-1.zip\node_modules\webpack\lib\webpack.js:55:30)
    at createWebpack (C:\dev\lec-migration\.yarn\$$virtual\@angular-devkit-build-webpack-virtual-0beaa59de8\0\cache\@angular-devkit-build-webpack-npm-0.900.0-rc.10-7017fe1d56-1.zip\node_modules\@angular-devkit\build-webpack\src\webpack-dev-server\index.js:20:36)
    at Object.runWebpackDevServer (C:\dev\lec-migration\.yarn\$$virtual\@angular-devkit-build-webpack-virtual-0beaa59de8\0\cache\@angular-devkit-build-webpack-npm-0.900.0-rc.10-7017fe1d56-1.zip\node_modules\@angular-devkit\build-webpack\src\webpack-dev-server\index.js:46:12)
    at SwitchMapSubscriber.project (C:\dev\lec-migration\.yarn\$$virtual\@angular-devkit-build-angular-virtual-d79a835d58\0\cache\@angular-devkit-build-angular-npm-0.900.0-rc.10-f83f279b13-1.zip\node_modules\@angular-devkit\build-angular\src\dev-server\index.js:191:32)
    at SwitchMapSubscriber._next (C:\dev\lec-migration\.yarn\cache\rxjs-npm-6.5.3-bc949c0a94-1.zip\node_modules\rxjs\internal\operators\switchMap.js:49:27)
    at SwitchMapSubscriber.Subscriber.next (C:\dev\lec-migration\.yarn\cache\rxjs-npm-6.5.3-bc949c0a94-1.zip\node_modules\rxjs\internal\Subscriber.js:66:18)
    at C:\dev\lec-migration\.yarn\cache\rxjs-npm-6.5.3-bc949c0a94-1.zip\node_modules\rxjs\internal\util\subscribeToPromise.js:7:24
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Other

Documentation suggested you add the fallowing into .yarnrc.yml nodeLinker: node-modules

When then produces the following error

Error: Something that got detected as your top-level application (because it doesn't seem to belong to any package) tried to access a package that is not declared in your dependencies       

Required package: @yarnpkg/fslib (via "@yarnpkg/fslib")
Required by: /C:/dev/lec-migration/node_modules/@yarnpkg/pnpify/lib/

[Jordan-Hall]

Angular appears not to be supporting PNP yet. Tracking https://github.com/angular/angular-cli/issues/12465

[ElbowBaggins]

According to this angular-cli comment this is no longer exactly the case.

It looks like native PnP mode support is not in yet, but @yarnpkg/pnpify or the node-modules linker strategy should (allegedly) be usable now.

Of course, I have no idea what this means for you guys, maybe this doesn't help you, but it's a thing now.

[Jordan-Hall]

@ElbowBaggins CLI still now working. getting command nx not found

[Jordan-Hall]

@FrozenPandaz

this issue is now relating to nx cli i believe https://github.com/nrwl/nx/blob/master/packages/cli/lib/init-local.tsit's looking directly in the node_module folder

[vaunus]

I am seeing the same issue here and disappointed NX CLI is breaking builds on yarn2 given it is now at version 2.1. Our project fails to build with default yarn2 settings, i.e. pnp enabled, but works as expected in node-modules mode.

> yarn nx serve

/project-path/.pnp.js:34448
    throw firstError;
    ^

Error: Qualified path resolution failed - none of the candidates can be found on the disk.

Source path: /project-path/node_modules/@nrwl/tao/index.js

Looks like this does indeed relate to the cli looking for tao in the node_modules directory as @Jordan-Hall indicated.

Is it possible to get a fix for this any time soon?

[Jordan-Hall]

I am seeing the same issue here and disappointed NX CLI is breaking builds on yarn2 given it is now at version 2.1. Our project fails to build with default yarn2 settings, i.e. pnp enabled, but works as expected in node-modules mode.

> yarn nx serve

/project-path/.pnp.js:34448
    throw firstError;
    ^

Error: Qualified path resolution failed - none of the candidates can be found on the disk.

Source path: /project-path/node_modules/@nrwl/tao/index.js

Looks like this does indeed relate to the cli looking for tao in the node_modules directory as @Jordan-Hall indicated.

Is it possible to get a fix for this any time soon?

Its annoying me now but im too ill to look. Ill carry on soon but they appears to be loads of files referencing direct file path grrr

[PetarKirov]

@FrozenPandaz @bekos what's the current status of Yarn v2 PnP support?

[derHodrig]

Is still in Progress https://github.com/angular/angular-cli/issues/16980

[Jordan-Hall]

@derHodrig this is internal of NX thats the issue atm

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs. If we missed this issue please reply to keep it active. Thanks for being a part of the Nx community! 🙏

[fguitton]

This is still of interest for ongoing projects

[hydrosquall]

https://github.com/nrwl/nx/blob/master/packages/cli/lib/init-local.ts

It looks like this step was solved in https://github.com/nrwl/nx/pull/3709

Is there a checklist of any other known current issues besides references to the string node_modules? After excluding test files (.spec.ts), it looks like there are ~79 references to node_modules (some of them strings) spread across several files remaining.

https://sourcegraph.com/search?q=repo:%5Egithub%5C.com/nrwl/nx%24+node_modules+lang:typescript++-file:.*.spec%5C.ts&patternType=literal

[KyleBastien]

We've been running an Nx repo with Yarn v2 PNP on loose mode for a few months now, and so far most things have worked for us.

The biggest exception is the nx migrate latest script, which just outright fails on Yarn v2 PNP. My workaround so far has been to switch Yarn v2 to use the node_modules folder, run the migrate latest script, and then switch back to using PNP before committing the result.

I would assume the migrate latest script's problem stem from something either not declaring a dependency correctly, or there being a string referencing a node_modules path. I'm just not sure which out of the above ~79 it might be.

[srosato]

@KyleBastien I wonder how you made it work in loose mode, I've added pnpMode: loose to my .yarnrc file, re-ran yarn install, but I still get an error when running

command: yarn nx affected:test

error:

(node:45) [MODULE_NOT_FOUND] Error: @nrwl/devkit tried to access rxjs, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

[KyleBastien]

@KyleBastien I wonder how you made it work in loose mode, I've added pnpMode: loose to my .yarnrc file, re-ran yarn install, but I still get an error when running

command: yarn nx affected:test

error:

(node:45) [MODULE_NOT_FOUND] Error: @nrwl/devkit tried to access rxjs, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

Note: All of the @nrwl package extensions here are unnecessary after Nx 12.6.2, because they were added to the respective package.json's

You can fix this by utilizing packageExtensions in your .yarnrc.yml file. Mine currently looks like:

packageExtensions:
  # unnecessary after 12.6
  '@nrwl/tao@*':
    dependencies:
      yargs: '15.4.1'
  # unnecessary after 12.6
  '@nrwl/jest@*':
    dependencies:
      identity-obj-proxy: '3.0.0'
  # unnecessary after 12.6
  '@nrwl/devkit@*':
    dependencies:
      rxjs: 6.6.6
  babel-plugin-const-enum@*:
    dependencies:
      '@babel/generator': ^7.9.6

Note: These should probably be added to the package.json's of the respective @nrwl packages. But Yarn does give you the tools to declare the dependencies yourself, so you're unblocked.

[srosato]

Thank you, trying it out, although I agree that we should not have to do this

[marckassay]

My workaround so far has been to switch Yarn v2 to use the node_modules folder, run the migrate latest script, and then switch back to using PNP before committing the result.

@KyleBastien You mentioned "switch", what exactly do you mean? I assume setting nodeLinker: node-modules in .yarnrc.yml, but anything else? And also, is nx installed globally for this to work? Thanks.

[KyleBastien]

@marckassay Here's the steps I follow to get this to work:

1. Remove pnpMode: loose from .yarnrc.yml
2. Add nodeLinker: node-modules to .yarnrc.yml
3. Run yarn to install all packages in node_modules
4. Run node ./node_modules/@nrwl/tao/index.js migrate latest
5. Run yarn again to install any changed that the migrate script made to package.json
6. Run node ./node_modules/@nrwl/tao/index.js migrate --run-migrations
7. Remove the migrations.json file
8. Remove nodeLinker: node-modules from .yarnrc.yml
9. Add pnpMode: loose to .yarnrc.yml
10. Run yarn again to remove the node_modules directory, and add back the .pnp.cjs file

[Lonli-Lokli]

Yarn 3 is already available...

[KyleBastien]

Edit: This is now fixed in Yarn v3.0.2, so using the link protocol is no longer necessary.

Yeap, Yarn v3 is unfortunately busted out of the box with an Nx repo. You can see the issue on the Yarn repo here: https://github.com/yarnpkg/berry/issues/3117.

You can use the link: protocol in Yarn to workaround this issue. By adding something like this to your package.json, dependencies:

"@name/your-library": "link:./packages/your-library/src",

Followed by running yarn. This is an unfortunate workaround, because you have to now maintain all your libraries in your package.json. A custom generator could probably be made to help automate this process.

This issue is caused by this: https://github.com/microsoft/TypeScript/pull/35206#discussion_r683139533 getPnpPath function in the patched version of TypeScript that Yarn v3 uses. Hopefully once this is fixed, references will work again with Yarn v3, and the link: workaround won't be necessary.

[KyleBastien]

You can use the link: protocol in Yarn to workaround this issue. By adding something like this to your package.json, dependencies:

Note this issue is fixed in Yarn v3.0.2, so using the link protocol isn't necessary anymore for your packages.

[benpsnyder]

Y'all might be interested in feat(core): implement Yarn v3

[sledorze]

@benpsnyder when is this expected to happen?

[ryanyue123]

@marckassay Here's the steps I follow to get this to work:

1. Remove pnpMode: loose from .yarnrc.yml
2. Add nodeLinker: node-modules to .yarnrc.yml
3. Run yarn to install all packages in node_modules
4. Run node ./node_modules/@nrwl/tao/index.js migrate latest
5. Run yarn again to install any changed that the migrate script made to package.json
6. Run node ./node_modules/@nrwl/tao/index.js migrate --run-migrations
7. Remove the migrations.json file
8. Remove nodeLinker: node-modules from .yarnrc.yml
9. Add pnpMode: loose to .yarnrc.yml
10. Run yarn again to remove the node_modules directory, and add back the .pnp.cjs file

i was able to get it working by following these steps, but i noticed a lot of the default nrwl executors did not work (ex: @nrwl/jest, @nrwl/lint). I just had to default to running jest --config ./myconfig. were you able to find a solution for this @KyleBastien ?

[KyleBastien]

@ryanyue123 I think that Nx has recently added features for some of these executors that uses swc. Since swc runs native code, it requires an install to trigger downloading that native code associated with the correct platform. I haven't figured out away around this besides running yarn (not zero-install) to ensure those native bits get downloaded, into the unplugged folder.

Maybe @benpsnyder can give an update on their PR to implement first class support for yarn v3, that would probably be the "right" fix to this, but I'm not sure if that PR supports zero-install or not.

I think at least one change that could be made here would be to move this if statement: https://github.com/nrwl/nx/blob/541d4e241bacc8097842fcff9cc8c5c5a743552a/packages/eslint-plugin-nx/src/resolve-workspace-rules.ts#L40

to here: https://github.com/nrwl/nx/blob/bafabb3a11c1d9084e114569f700cd793aac199f/packages/eslint-plugin-nx/src/index.ts#L30

Which would prevent the resolve-workspace-rules.ts file from importing from @swc-node/register/register unless workspace lint rules are actually used. This would solve this problem for the ESLint plugin at least in the case that you don't use workspace lint rules.

[ryanyue123]

@ryanyue123 I think that Nx has recently added features for some of these executors that uses swc. Since swc runs native code, it requires an install to trigger downloading that native code associated with the correct platform. I haven't figured out away around this besides running yarn (not zero-install) to ensure those native bits get downloaded, into the unplugged folder.

Maybe @benpsnyder can give an update on their PR to implement first class support for yarn v3, that would probably be the "right" fix to this, but I'm not sure if that PR supports zero-install or not.

I think at least one change that could be made here would be to move this if statement:

https://github.com/nrwl/nx/blob/541d4e241bacc8097842fcff9cc8c5c5a743552a/packages/eslint-plugin-nx/src/resolve-workspace-rules.ts#L40

to here:

https://github.com/nrwl/nx/blob/bafabb3a11c1d9084e114569f700cd793aac199f/packages/eslint-plugin-nx/src/index.ts#L30

Which would prevent the resolve-workspace-rules.ts file from importing from @swc-node/register/register unless workspace lint rules are actually used. This would solve this problem for the ESLint plugin at least in the case that you don't use workspace lint rules.

Thanks for the reply- i'll probably stick to running my own commands for now until nx officially supports yarn3. hopefully it's soon🤞🏻🤞🏻🤞🏻🤞🏻

[BigForNothing]

When running migrate, the command fails due to trying to use the -W flag from yarn v1

yarn nx migrate latest
Fetching meta data about packages.
It may take a few minutes.
Fetching @nrwl/workspace@latest

 ERROR  The migrate command failed.

Command failed: yarn add -W @nrwl/workspace@latest

In yarn v3 (and i believe v2), -W isn't used.

[MrChrisRodriguez]

Has anyone managed to get globally installed yarn to recognize Nx modules in PnP mode? The local yarn nx works well enough so far, but I'm curious if we can get global nx working in concert, too.

Related #8315

[nemonemi]

@MrChrisRodriguez how did you install NX globally? As far as I remember, Yarn doesn't support global packages.

[MrChrisRodriguez]

@nemonemi iirc both Yarn and Nx recommend using npm to install them globally.

1. npm i -g yarn
2. npm i -g nx

Both Yarn and Nx have mechanisms for checking if your cwd is configured for them and inherits that configuration. Global Nx hasn't implemented the PnP API, though, so you have to call it within the yarn context yarn nx ... or do what I did and just create the alias: alias ynx='yarn nx'

[giovai]

Does Nx 12.9 support Yarn 3 PnP?

[ild0tt0re]

I'm trying to switch to yarn 3.2.1 with PnP mode but despite having installed the missing package next-compose-plugins, it keeps giving me the same error:

yarn run nx serve hanami

> nx run hanami:serve:development

info  - Loaded env from apps/hanami/.env.development
info  - Loaded env from apps/hanami/.env
error - Failed to load next.config.js, see more info here https://nextjs.org/docs/messages/next-config-error
Your application tried to access next-compose-plugins, but it isn't declared in your dependencies; this makes the require call ambiguous and unsound.

Required package: next-compose-plugins
Required by: /Users/angeloca/dev/fe.monorepo/apps/hanami/

Require stack:
- /Users/angeloca/dev/fe.monorepo/apps/hanami/next.config.js

[Lonli-Lokli]

Is there any work in progress from Nx team? Angular seems like supports almost everything while task is still not closed

[ambeur]

Looking forward to this feature too!

[Lonli-Lokli]

Note: All of the @nrwl package extensions here are unnecessary after Nx 12.6.2, because they were added to the respective package.json's

@KyleBastien @JamesHenry I got same error without Nx with just angular-eslint. Should it be fixed there or here?

Required package: @angular-devkit/architect (via "@angular-devkit\architect")
Required by: @nrwl/devkit@npm:13.1.3 (via D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@nrwl-devkit-npm-13.1.3-a349b36d70-1cfd19866f.zip\node_modules\@nrwl\devkit\src\utils\)

Require stack:
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@nrwl-devkit-npm-13.1.3-a349b36d70-1cfd19866f.zip\node_modules\@nrwl\devkit\src\utils\convert-nx-executor.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@nrwl-devkit-npm-13.1.3-a349b36d70-1cfd19866f.zip\node_modules\@nrwl\devkit\index.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\__virtual__\@angular-eslint-builder-virtual-c8ba735627\0\cache\@angular-eslint-builder-npm-14.0.0-93f7fe6dc0-27e23a86bf.zip\node_modules\@angular-eslint\builder\dist\compat.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-devkit-architect-npm-0.1400.5-03eb80d39b-afb57021b2.zip\node_modules\@angular-devkit\architect\node\node-modules-architect-host.js     
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-devkit-architect-npm-0.1400.5-03eb80d39b-afb57021b2.zip\node_modules\@angular-devkit\architect\node\index.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-cli-npm-14.0.5-eccde1fd20-5bc4ae64b3.zip\node_modules\@angular\cli\src\command-builder\architect-base-command-module.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-cli-npm-14.0.5-eccde1fd20-5bc4ae64b3.zip\node_modules\@angular\cli\src\command-builder\architect-command-module.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-cli-npm-14.0.5-eccde1fd20-5bc4ae64b3.zip\node_modules\@angular\cli\src\commands\build\cli.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-cli-npm-14.0.5-eccde1fd20-5bc4ae64b3.zip\node_modules\@angular\cli\src\command-builder\command-runner.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-cli-npm-14.0.5-eccde1fd20-5bc4ae64b3.zip\node_modules\@angular\cli\lib\cli\index.js
- D:\gitlab\cv-app\frontend\cv-ui\.yarn\cache\@angular-cli-npm-14.0.5-eccde1fd20-5bc4ae64b3.zip\node_modules\@angular\cli\lib\init.js

And here is why

PS D:\gitlab\cv-app\frontend\cv-ui> yarn why @nrwl/devkit
├─ @angular-eslint/builder@npm:14.0.0
│  └─ @nrwl/devkit@npm:13.1.3 (via npm:13.1.3)
│
└─ @angular-eslint/builder@npm:14.0.0 [f4bce]
   └─ @nrwl/devkit@npm:13.1.3 (via npm:13.1.3)

[JamesHenry]

@Lonli-Lokli I can't comment about yarn 2/pnp because I am not familiar with it, but this has flagged up that I forgot to update the dependency on @nrwl/devkit for angular-eslint v14 so thank you for raising that, I will release a patch today

[snebjorn]

I think @nrwl/devkit also need to fix their dependency to @angular-devkit/architect as I'm getting this error when trying to lint an angular project using @angular-eslint@14.0.2.

❯ yarn lint
An unhandled exception occurred: @nrwl/devkit tried to access @angular-devkit/architect, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

Required package: @angular-devkit/architect (via "@angular-devkit\architect")
Required by: @nrwl/devkit@virtual:15288c7928c7c18c53a14dcfc084d0ce97806beb2231966887b0044d44e6502472ae028c1349166fbbbb635fc5f5cd532021a516d101259b0db620cd6b2b98a9#npm:14.4.2 (via D:\Users\dk8EskDi\Source\ng14-pnpm\.yarn\__virtual__\@nrwl-devkit-virtual-824b188259\0\cache\@nrwl-devkit-npm-14.4.2-4aa707a2fe-ed4c619803.zip\node_modules\@nrwl\devkit\src\utils\)

❯ yarn why @nrwl/devkit 
├─ @angular-eslint/builder@npm:14.0.2
│  └─ @nrwl/devkit@npm:14.4.2 (via npm:^14.2.4)
│
└─ @angular-eslint/builder@npm:14.0.2 [78dc4]
   └─ @nrwl/devkit@npm:14.4.2 [15288] (via npm:^14.2.4 [15288])

Workaround:

#.yarnrc.yml
packageExtensions:
  "@nrwl/devkit@*":
    dependencies:
      "@angular-devkit/architect": "*"

[MalikBagwala]

@AgentEnder @FrozenPandaz Any updates on this? this feature will make CI jobs very fast and efficient since you dont need to run yarn install on any step (which is a pain considering the number of packages in a mono-repo)

[tyteen4a03]

+1 - yarn 1 has been discontinued for 3 years now.

[muratbeser]

https://github.com/angular/angular-cli/issues/16980

[quicksilverr]

Hey, any updates regarding this? Does yarn 2 now work with nx?

[Peeja]

To be clear, as I understand it, Nx absolutely supports Yarn 2+, just not PnP mode. PnP is one of the main developments in Yarn 2, but the traditional node_modules mode is still very much supported.

It would be fantastic to have PnP support in Nx, but it's not as dire as only supporting Yarn 1, which as @tyteen4a03 says is no longer in active development.

[tyteen4a03]

@Peeja I struggled to get Yarn2 to work even in node linking mode.

[Peeja]

@tyteen4a03 Ah, that seems more concerning. I haven't tried myself in some time, so I was going by the comments above. But I wonder if conflating the issues is part of what keeps this from moving forward.

[AgentEnder]

iirc yarn2 with node modules linker should absolutely work at this point. If anyone is actively experiencing issues in latest please open a new report.

[slikts]

When getting used to the zero-installs workflow with Yarn PnP, switching back to node_modules is a bit painful, so it'd be really nice to have this feature.

[ild0tt0re]

Is there any roadmap on the work required to implement this feature?

[doughlass]

Is there any roadmap on the work required to implement this feature?

Apparently not.

[vire]

FYI 16.2.1 works for me with yarn2 - tho I'm using own webpack via run command, but then the next in 16.3.2 is broken as it calls npx next build

[nemonemi]

@vire when you say, it worked for you with yarn2, do you mean the Plug N Play version or the node_modules version?