search

microsoft / rushstack

Monorepo for tools developed by the Rush Stack community
https://rushstack.io/
Other
5.9k stars 595 forks source link

[rush] Feature request for RUSH_TEMP_FOLDER support when pnpm useWorkspaces is true #4974

Open timmydo opened 1 week ago

timmydo commented 1 week ago

Summary

We're trying to reduce git clean time for our developers. We have a C# repo used by hundreds of developers that invokes rush build to build 80+ rush projects. Long story short, git clean is part of their workflow due to a full build with caching being faster than incremental rebuilds when switching branches.

Currently, git clean can take 5 to 10 minutes to run. From what I can tell, git spends most of the time enumerating files in common/temp.

https://github.com/microsoft/rushstack/blob/681f718a04ffc148d4d25902e9453f5fdd1aacc0/libraries/rush-lib/src/logic/installManager/WorkspaceInstallManager.ts#L86

Repro steps

The symlinks created point to the virtual store in common/temp, which is indistinguishable from regular files to git. Just enumerating the files takes a while.

$ time find . | wc -l
263979

real    0m19.698s
user    0m1.531s
sys     0m11.717s

Expected result:

I'd like to specify an environment variable when running rush commands or installing rush that basically has the effect of RUSH_TEMP_FOLDER. Then I would set this environment variable to a location outside of the git repository.

iclanton commented 3 days ago

Do you want to just move the virtual store folder? We could probably include something similar to RUSH_PNPM_STORE_PATH for the virtual store. @D4N14L had some thoughts about this.

It'd require a fair bit of testing, though.

timmydo commented 3 days ago

I'm trying to get all written files by the build out of the git source directory. I'm using RUSH_PNPM_STORE_PATH successfully, so something similar to that would work.