asciidoctor / asciidoctor-vscode

AsciiDoc support for Visual Studio Code using Asciidoctor
Other
338 stars 97 forks source link

Perf issues in large repositories due to `workspace.findFiles` with `**` glob #809

Closed alexandruavadanii closed 1 year ago

alexandruavadanii commented 1 year ago

Those lines were introduced between v3.1.5 and v3.1.6 by: https://github.com/asciidoctor/asciidoctor-vscode/commit/862651d510515cfba35e08fe7586f99d7eb5b655 .

Downgrading to v3.1.5 helps, the number of rg instances spawned by vscode.workspace.findFiles is down to 1, but it's not zero, as it's also used in: https://github.com/asciidoctor/asciidoctor-vscode/blob/45a9162b41ced0b3d0aedb02f004b627e2f14fbd/src/features/antora/antoraSupport.ts#L180 Which is odd, considering that my Enable Antora support is not checked in the configuration.

FWIW, I don't even have an asciidoctorconfig or antora.yml.

Versions:

ggrossetie commented 1 year ago

Not only do those rg instances take a lot of time to finish, they make the virtual filesystem unusable while they run.

What does rg mean?

So when the extension is activated, the following code will spawn a couple of rg instances:

It should only activate when you open an AsciiDoc file. Is this the behavior you're noticing?

Which is odd, considering that my Enable Antora support is not checked in the configuration.

We try to find antora.yml files to ask the user if we should enable Antora support.

Not sure how we can solve this issue. How many sibling files do you have next to your AsciiDoc files? I guess we could provide an option to disable all features that rely on glob expression.

For reference, the search will ignore files.exclude-setting.

My vscode workspace contains a SCM (Clearcase) repository that exposes a virtual filesystem that contains very many objects (tens or hundreds of GB).

Is the virtual filesystem part of your VS Code workspace?

alexandruavadanii commented 1 year ago

Hi, Guillaume, and thank you for the quick reply.

What does rg mean?

rg is the ripgrep binary, a grep alternative that is supposed to be faster than the platform's grep binary. vscode uses rg via https://github.com/microsoft/vscode-ripgrep, so each call to vscode.workspace.findFiles spawns one (or more) rg processes that carry the actual search on the plaftorm.

It should only activate when you open an AsciiDoc file. Is this the behavior you're noticing?

Yes, it only activates when calling one of the extension commands or open an AsciiDoc file. I had my .adoc file already open and the active editor, so each time I reloaded vscode, I got new rg processes though, so it took me a while to figure out that my slow repository was caused by those rg processes.

We try to find antora.yml files to ask the user if we should enable Antora support.

Sounds good. Is there a particular path where those might reside or can they be anywhere in the repository? Because if they can be located anywhere, I don't see a better way to do it than the current approach. However, if they can only reside in the directory with .adoc or one of its parents, an up-search could be performed there too, like asciidoctorconfig was handled before https://github.com/asciidoctor/asciidoctor-vscode/commit/862651d510515cfba35e08fe7586f99d7eb5b655 - just look in each dir and move up till we reach one of the workspace root(s).

Not sure how we can solve this issue. How many sibling files do you have next to your AsciiDoc files? I guess we could provide an option to disable all features that rely on glob expression.

Not many siblings, so before https://github.com/asciidoctor/asciidoctor-vscode/commit/862651d510515cfba35e08fe7586f99d7eb5b655, searching for asciidoctorconfig is almost instant. If my .adoc file was in the repo root, things might be uglier though ... I agree, an option to avoid globs would be simpler to implement and it would definitely be enough for me, since I don't have either of these configuration files.

Is the virtual filesystem part of your VS Code workspace?

Yes, unfortunately, the way Clearcase works is that mixes regular files with virtual directories that hold all other versions of an object. So we can't (easily) separate the real files from the virtual objects.

For now, my workaround is to disable the extension on my regular vscode profile, open a new editor with a different profile that has the extension active and just load the .adoc in the second editor, which doesn't even have a workspace open, so no rg processes ...

Later edit: Just noticed I mentioned the extension version is 3.1.5 initially. The antora.yml search happens with 3.1.5 too, but my initial tests that triggered lots of rg processes were based on version 3.1.6, which uses ** for the config search.

ggrossetie commented 1 year ago

Sounds good. Is there a particular path where those might reside or can they be anywhere in the repository? Because if they can be located anywhere, I don't see a better way to do it than the current approach. However, if they can only reside in the directory with .adoc or one of its parents, an up-search could be performed there too, like asciidoctorconfig was handled before https://github.com/asciidoctor/asciidoctor-vscode/commit/862651d510515cfba35e08fe7586f99d7eb5b655 - just look in each dir and move up till we reach one of the workspace root(s).

The main issue with the previous approach is that it does not work in a Web environment (because it relies on fs and path Node.js modules).

antora.yml are located at the root of a documentation component:

📒 repository 
  📄 antora.yml 
  📂 modules 
    📂 ROOT 
      📁 attachments 
      📁 examples 
      📁 images 
      📁 pages 
      📁 partials 
      📄 nav.adoc 
    📂 named-module 
      📁 pages
      📄 nav.adoc 
  📁 packages

We are also considering using vscode.workspace.findFiles to located all image files in order to provide auto-completion. I need to think of an efficient solution... 🤔

alexandruavadanii commented 1 year ago

Would a mix between vscode.workspace.findFiles (but avoiding ** and instead specifying the directory to look into explicitly) and manual path split (using lastIndexOf('/')) work to traverse the file tree? This sounds like a common pattern, maybe it has been already been discussed in some github issue in the vscode repo?

ggrossetie commented 1 year ago

@alexandruavadanii could you please confirm if await vscode.workspace.findFiles('**/.{asciidoctorconfig.adoc,asciidoctorconfig}', null) is way slower than await vscode.workspace.findFiles('**/antora.yml', '/node_modules/', 100, cancellationToken.token) ?

I guess we should not specify the second argument (exclude) that might be the reason why findFiles('**/.{asciidoctorconfig.adoc,asciidoctorconfig}', null) is very slow.

A glob pattern that defines files and folders to exclude. The glob pattern will be matched against the file paths of resulting matches relative to their workspace. When undefined, default file-excludes (e.g. the files.exclude-setting but not search.exclude) will apply. When null, no excludes will apply.

alexandruavadanii commented 1 year ago

@ggrossetie I think so, but with a workspace this large, it's hard to say since I can't leave them run to completion to measure the total time.

Just for a simple test, I removed the null argument and rebuilt the extension, I have 11 instances of rg still running since 18 minutes ago when I opened an .adoc file and tried to generate a preview. My files.exclude pattern is non-empty, but it can't filter out most of the virtual filesystem objects, even if I tried improving it. So unfortunately, this might help, but is probably not enough.

$ pgrep -af 'antora|ascii'
92079 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/antora.yml -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* -g !/node_modules --no-ignore --no-config --no-ignore-global
92090 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/antora.yml -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* -g !/node_modules --no-ignore --no-config --no-ignore-global
92100 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/.{asciidoctorconfig.adoc,asciidoctorconfig} -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* --no-ignore --no-config --no-ignore-global
92110 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/.{asciidoctorconfig.adoc,asciidoctorconfig} -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* --no-ignore --no-config --no-ignore-global
92120 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/.{asciidoctorconfig.adoc,asciidoctorconfig} -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* --no-ignore --no-config --no-ignore-global
92130 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/.{asciidoctorconfig.adoc,asciidoctorconfig} -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* --no-ignore --no-config --no-ignore-global
92140 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/antora.yml -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* -g !/node_modules --no-ignore --no-config --no-ignore-global
92150 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/.{asciidoctorconfig.adoc,asciidoctorconfig} -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* --no-ignore --no-config --no-ignore-global
92161 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/antora.yml -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* -g !/node_modules --no-ignore --no-config --no-ignore-global
92171 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/.{asciidoctorconfig.adoc,asciidoctorconfig} -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* --no-ignore --no-config --no-ignore-global
92182 /redacted/path/.vscode-server-insiders/cli/servers/Insiders-7c1bff30eae1fcb2565c20a062a4fb44b7dd562b/server/node_modules/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g **/.{asciidoctorconfig.adoc,asciidoctorconfig} -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/*.{[ch]~,bak} -g !**/*.{o,d,a,lst} -g !**/*.{map,lss,sym} -g !**/*.{elf,eep} -g !/system/{gcc,.eccm,archive,jre,lost+found,Microsoft,source,vxworks}* --no-ignore --no-config --no-ignore-global

# rg instances have been running for 18 minutes
$ ps -p 92182 -o etime
    ELAPSED
      18:57
ggrossetie commented 1 year ago

I have 11 instances of rg still running since 18 minutes ago when I opened an .adoc file

You opened a single AsciiDoc file (and did nothing except opening it) and you have 11 instances of rg?

My files.exclude pattern is non-empty, but it can't filter out most of the virtual filesystem objects, even if I tried improving it.

👍🏻

Another, probably small, optimisation would be to find files on the workspace folder associated with the current AsciiDoc file. Currently, it finds files across all workspace folders.

Since Antora relies on convention, we could check that the path to the AsciiDoc file follows the following format: **/modules/**/pages/**/**/file.adoc or **/modules/**/partials/**/**/file.adoc before trying to locate antora.yml files.

Regarding .asciidoctorconfig, the method workspace.fs.stat might be available on both VS Code (desktop) and VS Code Web.

alexandruavadanii commented 1 year ago

I redid the experiment in a clean environment, as the previous test might have included a preview generation attempt and/or an attempt to generate the HTML:

Another, probably small, optimisation would be to find files on the workspace folder associated with the current AsciiDoc file. Currently, it finds files across all workspace folders.

Sounds good. I only have one workspace root, so it won't help in my case, but other multi-root large repositories might see a huge benefit from this.

Since Antora relies on convention, we could check that the path to the AsciiDoc file follows the following format: /modules//pages///file.adoc or /modules//partials///file.adoc before trying to locate antora.yml files.

That would be a very good check and should be easy to implement.

Based on my observations above, another possible optimization would be to store the state and/or result of the findFiles execution for antora.yml or .asciidoctorconf, something like not attempted yet, in progress, the path(s) found. That way we can at least avoid spawning new filesystem level searches each time an action happens. It's a compromise between memory usage and filesystem operations/some CPU cycles. My suggestion is basically memoization with an extra check for in progress.

stranljip commented 1 year ago

I have the same problem with numerous rg instances being created so that I have a load of >> 100 on my MacBook Pro. The problem disappears when I downgrade to 3.1.5. In my case I have a large repository with > 500 Maven modules and a handful of them contain AsciiDoc documents. The repository has been cloned on my local disk and there are no symlinks involved.

ggrossetie commented 1 year ago

I've addressed the performance issue on .asciidoctorconfig files. As mentioned above, for Antora we can implement several improvements. Please open a new issue.

ggrossetie commented 1 year ago

@alexandruavadanii @stranljip could you please try again with 3.1.7 ?

alexandruavadanii commented 1 year ago

@ggrossetie I can confirm that 3.1.7 fixes the multiple instance of rg searching for the asciidoctor config file. I still have the ones looking for antora.yml, but I think the committed fix works as intended.

I don't have an asciidoctorconfig file to confirm that the new mechanism actually works in that case, but I think you'll hear from others if it doesn't.

Thank you for the quick fix!

ggrossetie commented 1 year ago

I don't have an asciidoctorconfig file to confirm that the new mechanism actually works in that case, but I think you'll hear from others if it doesn't.

We have unit tests so I'm confident it's still working 😉 Thanks for creating a new issue.

stranljip commented 1 year ago

I just smoketested the new version, it seems to work better that 3.1.6 but I still have a heavy CPU load for a couple of seconds following any changes in an Asciidoc file with multiple instances of rg. Without further investigation it still "feels" less sluggish than before. I have to test more thoroughly next week and can give the results then. But anyway - thanks for addressing the issue so quick!

stranljip commented 1 year ago

@ggrossetie - the problem persists for me. I am not sure if this is because of the huge project with 100s of Maven modules or an isolated problem with the <10 directories with actual AsciiDoc/Antora content

ggrossetie commented 1 year ago

@stranljip could you please post the rg processes? I'm guessing it's related to https://github.com/asciidoctor/asciidoctor-vscode/issues/814

stranljip commented 1 year ago

Directly after opening an AsciiDoc file I have the following processes. The CPU load goes up to 100% on all CPUs for multiple seconds (between 5 - 15 sec) The effect occurs after every change in an open Asciidoc file and after opening another file. The directory in which Techdoc resides contains ~650 directories, whereof ~600 Maven modules (not sure if this might be relevant).

  502 23521 41158   0  8:02AM ??         0:02.46 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-explanations/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23522 41158   0  8:02AM ??         0:02.50 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23523 41158   0  8:02AM ??         0:02.33 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-admin/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23524 41158   0  8:02AM ??         0:02.19 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-howtos/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23525 41158   0  8:02AM ??         0:02.29 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-welcome/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23526 41158   0  8:02AM ??         0:02.20 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Web/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23527 41158   0  8:02AM ??         0:02.14 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Monitoring_Api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23528 41158   0  8:02AM ??         0:02.12 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-explanations/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23529 41158   0  8:02AM ??         0:02.20 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-admin/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23530 41158   0  8:02AM ??         0:02.07 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-welcome/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23531 41158   0  8:02AM ??         0:02.19 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-howtos/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23532 41158   0  8:02AM ??         0:01.99 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23533 41158   0  8:02AM ??         0:01.92 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Web/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23534 41158   0  8:02AM ??         0:01.87 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Monitoring_Api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23536 41158   0  8:02AM ??         0:01.94 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-explanations/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23537 41158   0  8:02AM ??         0:01.91 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-admin/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23538 41158   0  8:02AM ??         0:02.19 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-howtos/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23625 41158   0  8:02AM ??         0:01.94 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-welcome/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23634 41158   0  8:02AM ??         0:01.86 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Web/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23643 41158   0  8:02AM ??         0:01.90 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Monitoring_Api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23656 41158   0  8:02AM ??         0:01.93 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23990 41158   0  8:02AM ??         0:01.23 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-explanations/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23992 41158   0  8:02AM ??         0:01.22 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-admin/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23995 41158   0  8:02AM ??         0:01.27 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-howtos/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 23998 41158   0  8:02AM ??         0:01.28 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-cadenza-api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 24000 41158   0  8:02AM ??         0:01.31 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Techdoc/content/techdoc-welcome/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 24001 41158   0  8:02AM ??         0:01.26 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Web/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
  502 24002 41158   0  8:02AM ??         0:01.23 /Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar.unpacked/@vscode/ripgrep/bin/rg --files --hidden --case-sensitive --no-require-git -g /Cadenza_Monitoring_Api/modules/**/* -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db -g !**/.classpath -g !**/.project -g !**/.settings -g !**/.factorypath --no-ignore --follow --no-config --no-ignore-global
ggrossetie commented 1 year ago

We are using the modules/**/* glob to build a complete Antora catalog. How many Antora documentation component do you have? How many files do you have in each Antora documentation component?

/Techdoc/content/techdoc-welcome/modules/**/*
/Techdoc/content/techdoc-cadenza-howtos/modules/**/*
/Techdoc/content/techdoc-cadenza-api/modules/**/*
/Cadenza_Web/modules/**/*
/Cadenza_Monitoring_Api/modules/**/*
/Techdoc/content/techdoc-cadenza-explanations/modules/**/*
/Techdoc/content/techdoc-admin/modules/**/*
/Techdoc/content/techdoc-cadenza-howtos/modules/**/*
/Techdoc/content/techdoc-welcome/modules/**/*
/Cadenza_Web/modules/**/*
/Cadenza_Monitoring_Api/modules/**/*
/Techdoc/content/techdoc-cadenza-api/modules/**/*
/Techdoc/content/techdoc-cadenza-explanations/modules/**/*
/Techdoc/content/techdoc-admin/modules/**/*
/Techdoc/content/techdoc-cadenza-howtos/modules/**/*
/Techdoc/content/techdoc-cadenza-api/modules/**/*
/Techdoc/content/techdoc-welcome/modules/**/*
/Cadenza_Web/modules/**/*
/Cadenza_Monitoring_Api/modules/**/*

The directory in which Techdoc resides contains ~650 directories, whereof ~600 Maven modules (not sure if this might be relevant).

The Maven modules are not inside the modules directory right? If so, then this is not relevant. Anyway, please open a new issue so we can address this issue.