Packaging optimization

[ZtfCoder]

Too many markdown files result in a long packaging time. How can we optimize this time

[brc-dd]

How long does it take? Try disabling your plugins. It takes around 5 minutes to build a 1000 page site 👀

[ajiho]

Yes, when Vitepress encounters a large number of issues, it basically reaches a bottleneck, causing me to run npm run docs: dev and it feels like it's dead for a long time without any response. Is there still room for optimization?

I don't have a specific quantity test here, but there should be over 1000 MD files

[brc-dd]

@ajiho Does this happen on the latest vitepress version too? Do you have local search or twoslash enabled? Can you try disabling them and then testing once?

[ajiho]

vitepress version is 1.5.0 . When I turn off local search, the development server can start quickly .But we encountered new problems again . When I run npm run docs: build, it directly reports an error

⠧ building client + server bundles...Deprecation Warning: The legacy JS API is deprecated and will be removed in Dart Sass 2.0.0.

More info: https://sass-lang.com/d/legacy-js-api

⠋ building client + server bundles...Deprecation Warning: The legacy JS API is deprecated and will be removed in Dart Sass 2.0.0.

More info: https://sass-lang.com/d/legacy-js-api

⠹ building client + server bundles...
(!) Some chunks are larger than 500 kB after minification. Consider:
- Using dynamic import() to code-split the application
- Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
- Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
⠧ building client + server bundles...
<--- Last few GCs --->

[22692:00000184359E7000]   510796 ms: Mark-Compact 7517.9 (7718.1) -> 7504.5 (7718.8) MB, pooled: 1 MB, 6930.94 / 0.00 ms  (average mu = 0.234, 
current mu = 0.202) allocation failure; scavenge might not succeed
[22692:00000184359E7000]   528028 ms: Mark-Compact 7525.2 (7723.3) -> 7514.8 (7729.2) MB, pooled: 0 MB, 15847.00 / 2.88 ms  (average mu = 0.126, current mu = 0.080) allocation failure; scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
 1: 00007FF6EE41BD7B node::SetCppgcReference+16011
 2: 00007FF6EE384BC8 DSA_meth_get_flags+92344
 3: 00007FF6EEF12C31 v8::Isolate::ReportExternalAllocationLimitReached+65
 4: 00007FF6EEEFFA36 v8::Function::Experimental_IsNopFunction+2918
 5: 00007FF6EED4B6E0 v8::internal::StrongRootAllocatorBase::StrongRootAllocatorBase+31552
 6: 00007FF6EED48737 v8::internal::StrongRootAllocatorBase::StrongRootAllocatorBase+19351
 7: 00007FF6EED5DE34 v8::Isolate::GetHeapProfiler+7428
 8: 00007FF6EED5E6BA v8::Isolate::GetHeapProfiler+9610
 9: 00007FF6EED6F117 v8::Isolate::GetHeapProfiler+77799
10: 00007FF6EEA2D83B v8::internal::Version::GetString+435083
11: 00007FF68EFAD4BA

I found the same problem #issues/2870

I have tried your solution #issuecomment-1696752713

But still unable to solve the problem

When I reduce the number of markdown files to one tenth, I can successfully build it

@brc-dd Do you know the reason?

[brc-dd]

Can you try adjusting the buildConcurrency?

import { defineConfig } from 'vitepress'

export default defineConfig({
  buildConcurrency: 1 // this will be very slow, but please check if out of heap issue happens with this too
})

[ajiho]

After trying, it's still the same 'JavaScript heap out of memory' . I don't think this is an issue with Vitepress . NODE_OPTIONS=--max-old-space-size=8192 npm run docs:build When I adjusted 8182 to 10240, it compiled successfully When I add dozens of markdown files at this point, it will display 'JavaScript heap out of memory' again .So this is a headache problem Although the value of max old space size can be adjusted according to the increase in the number of markdown files, But this is not the final solution, as some platforms such as Vercel do not seem to support such adjustments And when building locally, your machine needs enough RAM . I think as you said, it is recommended to use other tools instead when building a large website . I also found some other reference links #error-javascript-heap-out-of-memory Perhaps the ultimate source is rollup It seems that both the Rollup and Vite teams are not very concerned about this issue

[ajiho]

By the way, the default theme was used

[Zhengqbbb]

If anyone needs to perform repairs and testing, can notify me. X-CMD currently has over 3000 pages, most of which are generated using dynamic routing.

After the recent version upgrade, we are experiencing the same build failure issue both locally machine and GitHub Actions. Currently we frozen old versions to ensure build success.

# freeze version
  vitepress: '1.4.1'
  vue: '3.4.38'

# script
NODE_OPTIONS='--max-old-space-size=8192' vitepress build