perf: skipping gitignore files when traverse workspace.

[IWANABETHATGUY]

This is my computer info

  System:
    OS: macOS 12.5.1
    CPU: (10) arm64 Apple M1 Max
    Memory: 315.63 MB / 32.00 GB
    Shell: 3.6.1 - /opt/homebrew/bin/fish

[IWANABETHATGUY]

for project https://github.com/web-infra-dev/rspack If I run time TARGET_PKG=@rspack/core FEATURE=scripts pnpm-shell-completion, this is the result:

prepare
build
dev
test
precompile-schema

________________________________________________________
Executed in  909.71 millis    fish           external
   usr time  157.10 millis  128.00 micros  156.97 millis
   sys time  762.42 millis  933.00 micros  761.49 millis

here is the profile

[IWANABETHATGUY]

You will find that the hot point is the reading directory which is slow IO. try to inspect how many directories do we read,

   WalkDir::new(&base_dir)
            .into_iter()
+            .inspect(|_| {
+                println!("1");
+            })

run TARGET_PKG=@rspack/core FEATURE=scripts pnpm-shell-completion | wc -l, the result shows below:

261030

It seems we search the whole node_modules and other directories that ignored by .gitignore, replace walkdir with ignore(which could ignore walk the directories that ignored by .gitignore ), rerun TARGET_PKG=@rspack/core FEATURE=scripts pnpm-shell-completion | wc -l, the result shows below:

16525

[IWANABETHATGUY]

performance: before

prepare
build
dev
test
precompile-schema

________________________________________________________
Executed in  909.71 millis    fish           external
   usr time  157.10 millis  128.00 micros  156.97 millis
   sys time  762.42 millis  933.00 micros  761.49 millis

after

precompile-schema
build
prepare
test
dev

________________________________________________________
Executed in  164.30 millis    fish           external
   usr time   39.68 millis  101.00 micros   39.58 millis
   sys time  130.44 millis  689.00 micros  129.75 millis

huge win

[IWANABETHATGUY]

Another reason we could do this is: almost no one has a scenario like having a package belonging to a workspace but being ignored by .gitignore

[IWANABETHATGUY]

I have tested it locally, the function works fine with this modification.