search

nrwl / nx-console

Nx Console is the user interface for Nx & Lerna.
https://nx.dev
MIT License
1.31k stars 213 forks source link

In yarn PNP the plugin keeps touching cache files, triggering reindexing #2062

Open prewk opened 8 months ago

prewk commented 8 months ago

Current Behavior

When I enable the plugin in WebStorm, it keeps causing reindexations over and over again. I just need to switch out of WebStorm (triggering a WebStorm save I suppose?) and come back, and - reindexation!

My colleague who uses the plugin in VS Code has a similar problem (but VSC doesn't auto-save).

Turning the plugin off, this problem disappears.

Expected Behavior

It shouldn't reindex.

Steps to Reproduce

I've converted our classic yarn project to a latest (v4) yarn PNP project with zero-installs.

Meaning, instead of a node_modules we have all the packages checked into git in .yarn/cache.

Starting WebStorm or VS Code with nx-console enabled. Change a file and bam - reindexing.

Failure Logs / Images / Videos

I'm monitoring changed files with chokidar:

chokidar "*.*" "**/*.*" -c "echo '{path}'"

With nx-console plugin turned off:

change:.idea/workspace.xml
change:.idea/aws.xml
.idea/aws.xml

With nx-console plugin turned on:

# constant pattern of change in the cache files over and over again, this is just an excerpt:

change:.yarn/cache/@storybook-angular-npm-7.5.3-cb80502f1a-fd51664361.zip
change:.yarn/cache/@testing-library-angular-npm-14.2.0-e1624ada75-e41e72edf7.zip
change:.yarn/cache/@testing-library-angular-npm-14.2.0-e1624ada75-e41e72edf7.zip
change:.yarn/cache/@schematics-angular-npm-17.2.3-1576f57e2c-a350fbbd91.zip
.yarn/cache/@schematics-angular-npm-17.2.3-1576f57e2c-a350fbbd91.zip
change:.nx/cache/project-graph.json
change:.nx/cache/file-map.json
.nx/cache/file-map.json

Environment

nx report (also tried on latest nx with same results):

   Node   : 18.17.1
   OS     : darwin-arm64
   yarn   : 4.1.1

   nx                 : 17.3.2
   @nx/js             : 17.3.2
   @nx/jest           : 17.3.2
   @nx/eslint         : 17.3.2
   @nx/workspace      : 17.3.2
   @nx/angular        : 17.3.2
   @nx/devkit         : 17.3.2
   @nx/eslint-plugin  : 17.3.2
   @nx/plugin         : 17.3.2
   @nx/storybook      : 17.3.2
   typescript         : 5.3.3
   ---------------------------------------
   Community plugins:
   @ngneat/spectator        : 14.0.0
   @ngrx/component          : 17.1.1
   @ngrx/component-store    : 17.1.1
   @ngrx/effects            : 17.1.1
   @ngrx/entity             : 17.1.1
   @ngrx/router-store       : 17.1.1
   @ngrx/schematics         : 17.1.1
   @ngrx/store              : 17.1.1
   @ngrx/store-devtools     : 17.1.1
   @storybook/angular       : 7.5.3
   @testing-library/angular : 14.2.0
   ---------------------------------------
MaxKless commented 8 months ago

hmm this shouldn't happen, you're right. Is nx/cache part of your .gitignore?

prewk commented 8 months ago

No, because we're using zero-install so all the dependencies are checked into git.

MaxKless commented 8 months ago

It seems to me like IntelliJ is picking up everything for indexing so it makes sense that it reindexes whenever a file in there changes. Adding it to .gitignore or manually exempting the folder from indexing seems like the easiest solution... Can you check whether that fixes it?

prewk commented 8 months ago

whenever a file in there changes.

But it doesn't change! You can sit and cmd/alt+tab between the IDE and something else doing nothing. And VS Code also has the problem.

Nothing is touching that directory (as evident by chokidar), only when nx-console is enabled, the touching starts.

.gitignore

Not an option unfortunately, the point is to not have it ignored in git

manually exempting

Doesn't change anything unfortunately.

MaxKless commented 8 months ago

hmmm it makes sense that things in .nx/cache (not yarn/cache) change though. Once nx console starts, it starts the Nx Language Server which does all kinds of calculations about your workspace. These nx/cache/project-graph.json files that chokidar logs are byproducts of that. So .nx/cache definitely shouldn't be checked into git and should be ignored.

As to the yarn files that change though, that's an interesting thing I'm not sure how it happens...

prewk commented 8 months ago

Ah yes, .nx is completely git ignored. It's .yarn/cache that's the problem here :)

MaxKless commented 8 months ago

I've been able to reproduce this and I'll have an improvement in a PR soon. I'm still not quite sure why the files in .yarn/cache keep changing but I've excluded them from the file watcher. That will solve a lot of performance issues but not the reindexing. I'm in contact with the webstorm team to figure out what's happening here - because .yarn/cache is marked as excluded, changes inside it shouldn't lead to reindexing.

prewk commented 8 months ago

Great!

because .yarn/cache is marked as excluded, changes inside it shouldn't lead to reindexing.

A note here - it's the plugin that performs the changes/touching. It's obvious if you monitor with chokidar (see my OP)

MaxKless commented 8 months ago

yeah I know. We read the files to figure out what dependencies contain generators & executors so we can show them in Nx Console. So it makes sense that Nx Console is involved. There's two questions: