Ignore "**/node_modules" very slow, more than 40s

[fkworld]

My project

A pnpm workspace monorepo, 16 packages & 1000+ files.

ls-lint v2.2.3 go go1.22.1 X:nocoverageredesign

Slow way

Use ignore "**/node_modules" in .ls-lint.yml

ls:
  .dir: kebab-case
  .js: kebab-case
  .jsx: kebab-case
  .cjs: kebab-case
  .mjs: kebab-case
  .config.js: kebab-case
  .ts: kebab-case
  .tsx: kebab-case
  .d.ts: kebab-case
  .test.ts: kebab-case
  .test.tsx: kebab-case
  .config.ts: kebab-case
  .md: kebab-case
  .mdx: kebab-case
  .ico: kebab-case
  .png: kebab-case

ignore:
  - .git
  - .husky
  - .vscode
  - "**/.turbo"
  - "**/dist"
  - "**/node_modules"
  - "**/README.md"

debug output

-----------------------------
statistics
-----------------------------
time: 45040 ms
files: 1083
file skips: 16
dirs: 277
dir skips: 33
=============================

full debug file slow.txt

In debug file, I find that ignore files is from line 8 to line 9945, maybe it's the reason why very slow?

Fast way

Use more specific glob pattern in .ls-lint.yml

ls:
  .dir: kebab-case
  .js: kebab-case
  .jsx: kebab-case
  .cjs: kebab-case
  .mjs: kebab-case
  .config.js: kebab-case
  .ts: kebab-case
  .tsx: kebab-case
  .d.ts: kebab-case
  .test.ts: kebab-case
  .test.tsx: kebab-case
  .config.ts: kebab-case
  .md: kebab-case
  .mdx: kebab-case
  .ico: kebab-case
  .png: kebab-case

ignore:
  - .git
  - .husky
  - .vscode
  - .turbo
  - dist
  - node_modules
  - README.md
  - "./{apps,packages}/*/.turbo"
  - "./{apps,packages}/*/dist"
  - "./{apps,packages}/*/node_modules"
  - "./{apps,packages}/*/README.md"

debug output

-----------------------------
statistics
-----------------------------
time: 67 ms
files: 1085
file skips: 16
dirs: 277
dir skips: 33
=============================

full debug file fast.txt

In debug file, I find that ignore files is from line 8 to line 57

My thoughts

I think may be all ignore files are compared when every file lint, causing the slowness.

If this is expected, it would be best to add a usage tip to the official website.

[loeffel-io]

thanks @fkworld for the report

can you please provide a time comparision from v2.2.3 and v2.3.0-beta.1 for the slow way 🙏

[loeffel-io]

any can you please provide if this also works?

  - "{apps,packages}/*/.turbo"
  - "{apps,packages}/*/dist"
  - "{apps,packages}/*/node_modules"
  - "{apps,packages}/*/README.md"

not sure if "./" works

[loeffel-io]

pretty sure i can optimize this, the issue is definitly that the filepaths will be added to the debug index in the slow way while only the directories will be added to the debug index in the fast way

[fkworld]

thanks @fkworld for the report

can you please provide a comparision from v2.2.3 and v2.3.0-beta.1 🙏

Use v2.2.3

Refer to the issue description at the top.

Use v2.3.0-beta.1

ls-lint v2.3.0-beta.1 go go1.22.5 X:nocoverageredesign

Slow way

debug output: (Pay attention to the time unit)

-----------------------------
statistics
-----------------------------
time: 47.055022s
paths: 1125
file skips: 16
dirs: 281
dir skips: 34
=============================

full debug file slow.txt

Fast way

debug output:

-----------------------------
statistics
-----------------------------
time: 44.163ms
paths: 1124
file skips: 16
dirs: 281
dir skips: 34
=============================

full debug file fast.txt

[loeffel-io]

amazing thank you!

[fkworld]

any can you please provide if this also works?

  - "{apps,packages}/*/.turbo"
  - "{apps,packages}/*/dist"
  - "{apps,packages}/*/node_modules"
  - "{apps,packages}/*/README.md"

not sure if "./" works

Both works.

[loeffel-io]

any can you please provide if this also works?

  - "{apps,packages}/*/.turbo"
  - "{apps,packages}/*/dist"
  - "{apps,packages}/*/node_modules"
  - "{apps,packages}/*/README.md"

not sure if "./" works

Both works.

great thanks