slorber commented 1 month ago

Docusaurus v3.6 is out with the Docusaurus Faster options 🥳 🎉

Docusaurus Faster

This is a meta-issue for the Docusaurus Faster project.

The goal is to modernize our infrastructure and greatly reduce the time and memory it takes to build a Docusaurus site in production mode, to fix this long-standing issue https://github.com/facebook/docusaurus/issues/4765.

yarn add @docusaurus faster

Before using these flags, you will need to add an extra package encapsulating all the opt-in dependencies:

It includes multiple experimental PRs adding new siteConfig.future.experimental_faster options:

swcJsLoader: Use SWC to transpile JS (instead of Babel).
swcJsMinimizer: Use SWC to minify JS (instead of Terser).
swcHtmlMinimizer: Use SWC to minify HTML and inlined JS/CSS (instead of html-minifier-terser).
lightningCssMinimizer: Use Lightning CSS to minify CSS (instead of cssnano and clean-css).
rspackBundler: Use Rspack to bundle your app (instead of webpack).
mdxCrossCompilerCache: Compile MDX files only once for both browser/Node.js environments instead of twice.

const config = {
  future: {
    experimental_faster: {
      optionName: true
    },
  },
}

We recommend turning them on all at once with this simple boolean shortcut:

const config = {
  future: {
    experimental_faster: true,
  },
}

But you can also turn them on/off one by one to troubleshoot potential problems.

Note that the rspackBundler option requires swcJsLoader, swcJsMinimizer, and lightningCssMinimizer to be enabled first.

It is possible that certain combinations of options do not work, and our CI doesn't cover all possible permutations (only true / false).

It also includes notable memory improvements:

Rspack takes much less memory to bundle your app
Fix important memory leak affecting i18n sites

What to expect

With experimental_faster: true, our Docusaurus website builds:

~ 3-4x faster on cold builds 🔥
~ same on hot re-builds (because Rspack doesn't support persistent cache yet)
the Rspack bundling consumes much less memory

Although rebuilds (with warm ./node_modules/.cache) are not faster (yet), we recommend you enable the experimental_faster: true option to future-proof your site against our new dependencies and report us potential problems you encounter.

Our core plugins/themes are all compatible with this new option, and we expect most community plugins to be compatible out-of-the-box. Some plugins might need minor adjustments to make them compatible with Rspack, and we'll help the community to make our ecosystem compatible with Rspack.

This option is relatively safe to use in production: we have deployed our own website with it already:

If your Docusaurus site is standard, it should be fine out of the box
If your Docusaurus site is more complex, make sure to test it thoroughly and give us feedback

For Docusaurus plugins authors

Some plugins implementing the configureWebpack() lifecycle hook might need to adjust their code to add support for Rspack.

I will write a guide for them in this sister issue: https://github.com/facebook/docusaurus/issues/10572

Upgrade PRs

Example upgrade PRs to look at, including some benchmarks.

Community websites:

Popular plugins:

Contributing to this issue

We want our community to share feedback on this Docusaurus Faster project.

Please try to adopt it as part of the v3.6+ / canary releases, and tell us how it works for you.

If you encounter problems, please let us know here, and we might create dedicated issues if needed.

This general issue is a great place to follow the Docusaurus Faster project overall, as we try to stabilize it and aim to make it the default for an upcoming major version.

sserrata commented 1 month ago

Thanks @slorber, sharing comparison results for our OpenAPI plugin site.

Cold Build No Optimization

[PERF] Build > en > Bundling with webpack - 41.58 seconds!
[PERF] Build > en > SSG > Load App renderer > Load server bundle - 8.49 ms
[PERF] Build > en > SSG > Load App renderer > Server bundle size = 17.241 MB
[PERF] Build > en > SSG > Load App renderer > Evaluate server bundle - 225.13 ms
[PERF] Build > en > SSG > Load App renderer - 234.15 ms
[PERF] Build > en > SSG > Generate static files - 3.26 seconds!
[PERF] Build > en > SSG - 3.50 seconds!
[PERF] Build > en > postBuild() - 10.58 ms
[PERF] Build > en > Broken links checker - 28.01 ms
[SUCCESS] Generated static files in "build".
[PERF] Build > en - 46.38 seconds!
[PERF] Build - 46.38 seconds!
[INFO] Use `npm run serve` command to test your build locally.
✨  Done in 51.97s.

Cold Build With Experimental Faster Enabled

[PERF] Build > en > Bundling with webpack - 23.44 seconds!
[PERF] Build > en > SSG > Load App renderer > Load server bundle - 7.16 ms
[PERF] Build > en > SSG > Load App renderer > Server bundle size = 17.767 MB
[PERF] Build > en > SSG > Load App renderer > Evaluate server bundle - 222.76 ms
[PERF] Build > en > SSG > Load App renderer - 230.31 ms
[PERF] Build > en > SSG > Generate static files - 1.82 seconds!
[PERF] Build > en > SSG - 2.06 seconds!
[PERF] Build > en > postBuild() - 9.49 ms
[PERF] Build > en > Broken links checker - 27.01 ms
[SUCCESS] Generated static files in "build".
[PERF] Build > en - 26.79 seconds!
[PERF] Build - 26.79 seconds!
[INFO] Use `npm run serve` command to test your build locally.
✨  Done in 32.52s.

Edit from @slorber: results above are still based on Webpack.

See https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/pull/1003 for the real upgrade.

Benchmark shows demo site building:

2.7x faster on cold builds
1.2x slower (for now) on warn builds

slorber commented 1 month ago

Thanks for reporting these early results @sserrata

Note that the Rspack PR hasn't been merged yet and if you use canary today you are still using Webpack.

For plugin authors, it's possible that Rspack will not work immediately out of the box. Some plugins will have to adjust their code to add Rspack support, eventually requiring a minimum version of Docusaurus v3.6+ to do so.

Follow this issue to know when a guide for plugin authors is ready: https://github.com/facebook/docusaurus/issues/10572

slorber commented 1 month ago

The Rspack support has been merged: https://github.com/facebook/docusaurus/pull/10402

You can try canary 0.0.0-6101 or above to benefit from all the upcoming perf improvements.

Here's an example PR upgrading the React Native website: https://github.com/facebook/react-native-website/pull/4268

How to benchmark

One way to benchmark this is to use hyperfine with an env variable:

const config = {
  future: {
    experimental_faster: (process.env.DOCUSAURUS_FASTER ?? 'true') === 'true',
  },
}

For cold builds:

hyperfine --prepare 'yarn clear' --runs 5 'DOCUSAURUS_FASTER=false yarn build' 'DOCUSAURUS_FASTER=true yarn build'

For warm rebuilds:

DOCUSAURUS_FASTER=false yarn build
hyperfine --runs 5 'DOCUSAURUS_FASTER=false yarn build' 'DOCUSAURUS_FASTER=true yarn build'

Troubleshooting slow builds

It's possible that enabling this new option does not give you the expected speed increase.

Perf logger

You can get a more verbose performance logger with the DOCUSAURUS_PERF_LOGGER=true env variable, giving you the time it takes to execute each step:

Rsdoctor

Another option is to use Rsdoctor, which works for both webpack and Rspack.

It can tell you which loader/plugin/phase is slow while bundling your app.

For many Docusaurus sites (including ours), the slow loaders are postcss/css-loader, mdx loader, and the image loaders of our ideal image plugin. Despite these slow loaders, Docusaurus Faster builds 3x faster.

It is possible that a Docusaurus plugin you use slows down your build. Rsdoctor might be able to give you insight on that.

We'll probably release an official Rsdoctor plugin for Docusaurus, but in the meantime here's a Docusaurus plugin we built to integrate it into our own site: https://github.com/facebook/docusaurus/blob/main/website/src/plugins/rsdoctor/RsdoctorPlugin.ts

o-l-a-v commented 3 weeks ago

Tested with a relatively large internal documentation site.

PS > Get-ChildItem -File -Recurse | Group-Object -Property 'Extension' | Sort-Object -Property 'Count' -Descending | Format-Table -Property 'Count', 'Name'

Count Name
----- ----
  748 .md
  553 .png
   71 .jpg
   24 .json
   10 .webp
    8 .svg
    5 .order
    4 .py
    3 .drawio
    3 .html
    3 .pdf
    2 .mdx
    2 .xml
    1 .xlsx
    1
    1 .avif
    1 .zip

PS >

Locally, Windows, Yarn v4.5.1, NodeJS v20.18.0. Used following command in PowerShell for timing the build:

yarn docusaurus clear; $Stopwatch = [System.Diagnostics.Stopwatch]::StartNew(); yarn build; $Stopwatch.Stop(); $Stopwatch.Elapsed.ToString()

3.5.2: 00:02:32.9615610
Canary with experimental_faster: true: 00:01:10.1179920

We're pretty stoked on that speedup. 😊

sedghi commented 3 weeks ago

which version of docusaurus comes with this experimental config? since i can't get it to run in 3.5.2

[ERROR] Error: These field(s) ("future.experimental_faster",) are not recognized in docusaurus.config.js.
If you still want these fields to be in your configuration, put them in the "customFields" field.
See https://docusaurus.io/docs/api/docusaurus-config/#customfields
    at validateConfig (/Users/alireza/dev/admin/cornerstone3D.git.worktrees/beta/node_modules/@docusaurus/core/lib/server/configValidation.js:299:15)
    at loadSiteConfig (/Users/alireza/dev/admin/cornerstone3D.git.worktrees/beta/node_modules/@docusaurus/core/lib/server/config.js:39:62)
    at async Promise.all (index 1)
    at async loadContext (/Users/alireza/dev/admin/cornerstone3D.git.worktrees/beta/node_modules/@docusaurus/core/lib/server/site.js:37:97)
    at async loadSite (/Users/alireza/dev/admin/cornerstone3D.git.worktrees/beta/node_modules/@docusaurus/core/lib/server/site.js:136:21)
    at async createReloadableSite (/Users/alireza/dev/admin/cornerstone3D.git.worktrees/beta/node_modules/@docusaurus/core/lib/commands/start/utils.js:50:16)
    at async Command.start (/Users/alireza/dev/admin/cornerstone3D.git.worktrees/beta/node_modules/@docusaurus/core/lib/commands/start/start.js:22:28)
[INFO] Docusaurus version: 3.5.2

sedghi commented 3 weeks ago

Answering myself, it is 3.5.2-canary-6125, I'll update the speed benchmarks soon

I'm seeing

dev: 15 seconds -> 2 seconds
build: +3 mins before giving any error (I'm upgrading to 3.x from 2.x) to 10 seconds with relevant mdx erros

Nice job

Update: build is down from +3 minutes to 30 seconds

societymartingale commented 2 weeks ago

I ran some informal benchmarks on a Macbook M3 Pro building our Docusaurus site. I cleared the cache after each build. Seeing a 2.5x speedup.

existing build (s)
-----------------
70.2
72.2
72.0
71.5
72.5
average: 71.7s

experimental_faster build (s)
-----------------------------
31.7
27.4
27.0
27.6
27.8
average: 28.3s

speedup = 71.7 / 28.3 = 2.5

Our CI runners are significantly slower than my Macbook, so I'm estimating that it will reduce the build time by 2 minutes in the CI pipeline.

lebalz commented 2 weeks ago

Wanted to test it on a docusaurus page that uses stage 3 decorators. Is it (or will it be) possible to customize the configuration of swc? (I have no experience so far with swc, but what i understand from the docs, the following should work:

{
  "jsc": {
    "parser": {
      "syntax": "typescript",
      "decorators": true
    }
  }
}

)

slorber commented 2 weeks ago

I'm not 100% sure but maybe adding a .swcrc would work? https://swc.rs/docs/configuration/swcrc

lebalz commented 2 weeks ago

Thanks for the advice @slorber . I tried it, but somehow swc does not pick up the .swcrc configuration.

Here is a little repro:

https://codesandbox.io/p/devbox/sharp-bird-jd7tf5

With future: { experimental_faster: { swcJsLoader: false } } it compiles and works as intended (click the Click Me Button and check the console log to see it working).

The used code for the decorators can be compiled by the swc playground when enabling the decorators.

decorated code example

```ts type AccessorDescriptor = { get: () => number; set: (value: number) => void; }; type DecoratorContext = { name: string; kind: string; }; function accessorDecorator( { get, set }: AccessorDescriptor, { name, kind }: DecoratorContext ) { if (kind === "accessor") { return { init(initialValue: any) { return initialValue ?? 0; }, get() { const value = get.call(this); // Additional logic can be added here return value; }, set(newValue: any) { const oldValue = get.call(this); console.log("set", name, "=", newValue); // Additional logic can be added here set.call(this, newValue); }, }; } } class ExampleClass { @accessorDecorator accessor exampleField: number = 1; } export default ExampleClass; ```

Maybe the options must be passed to swc explicitely 👉 https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-faster/src/index.ts#L25-L34 ?

Update 5.11.2024

The workaround from https://github.com/facebook/docusaurus/issues/10556#issuecomment-2454482074 works:

const config = {
  future: {
    experimental_faster: {
      swcJsLoader: false, /* set it to false here */
      swcJsMinimizer: true,
      swcHtmlMinimizer: true,
      lightningCssMinimizer: true,
      rspackBundler: true,
      mdxCrossCompilerCache: true,
    },
  },
  webpack: {
    jsLoader: (isServer) => ({
      loader: 'builtin:swc-loader', // (only works with Rspack)
      options: {
        ...require("@docusaurus/faster").getSwcLoaderOptions({isServer}),
        decorators: true
      },
    }),
  },
}

Thanks @slorber

slorber commented 2 weeks ago

I don't know, will need to investigate @lebalz

Note that you might still be able to use the old custom js loader option mentioned here: https://github.com/facebook/docusaurus/issues/4765#issuecomment-841135926

Didn't try, but pseudo-code:

const config = {
  webpack: {
    jsLoader: (isServer) => ({
      loader: 'builtin:swc-loader', // (only works with Rspack)
      options: require("@docusaurus/faster").getSwcLoaderOptions({isServer}),
    }),
  },
}

slorber commented 2 weeks ago

Docusaurus v3.6 is out with the Docusaurus Faster options 🥳 🎉

https://docusaurus.io/blog/releases/3.6

o-l-a-v commented 2 weeks ago

Would it be possible to add an option in create-docusaurus to add @docusaurus/faster and enable it by adding future: {experimental_faster: true} to the config?

https://docusaurus.io/docs/api/misc/create-docusaurus

Maybe --docusaurus-faster with default value false?

It'd also be useful to disable using Docusaurus Faster whenever it becomes the default.

slorber commented 2 weeks ago

@o-l-a-v I'm not sure it's necessary. Depending on initial feedback I plan to make it soon the default of our init template to increase adoption, feedback, bugfixes, and finally making it the new default infrastructure.

ruibaby commented 1 week ago

Hi, I found a new problem after turning on faster mode. This may be related to rspack. This problem only occurs in the development environment (pnpm start).

❯ DOCUSAURUS_PERF_LOGGER=true pnpm start

> @halo-dev/docs@0.0.0 start /Users/ryanwang/Workspace/github/ruibaby/docs
> docusaurus start

  ----------------------------------------------------------------------------------------------------------------------------------------------- 

                                                    Update available 3.6.0-canary-6132 → 3.6.0                                                    

                                To upgrade Docusaurus packages with the latest version, run the following command:                                
       `npm i @docusaurus/core@latest @docusaurus/faster@latest @docusaurus/plugin-client-redirects@latest @docusaurus/preset-classic@latest      
     @docusaurus/theme-classic@latest @docusaurus/theme-common@latest @docusaurus/theme-mermaid@latest @docusaurus/module-type-aliases@latest`    

  ----------------------------------------------------------------------------------------------------------------------------------------------- 

[INFO] Starting the development server...
[PERF] Load site > Load context - 115.72 ms - (48mb -> 64mb)
[PERF] Load site > Load plugins > Init plugins - 191.18 ms - (64mb -> 78mb)
[PERF] Load site > Load plugins > Load plugins content > Load classic@default > translatePluginContent() - 8.34 ms - (79mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load pages@default > loadContent() - 9.00 ms - (79mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load pages@default - 19.53 ms - (79mb -> 80mb)
[PERF] Load site > Load plugins > Load plugins content > Load classic@default > getDefaultCodeTranslationMessages() - 11.98 ms - (81mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load classic@default - 20.62 ms - (79mb -> 81mb)
[PERF] Load site > Load plugins > Load plugins content > Load docs@default > loadContent() - 138.61 ms - (79mb -> 86mb)
[PERF] Load site > Load plugins > Load plugins content > Load docs@default > contentLoaded() - 39.50 ms - (87mb -> 95mb)
[PERF] Load site > Load plugins > Load plugins content > Load docs@default - 180.91 ms - (79mb -> 95mb)
[PERF] Load site > Load plugins > Load plugins content - 181.09 ms - (79mb -> 95mb)
[PERF] Load site > Load plugins - 374.37 ms - (64mb -> 95mb)
[PERF] Load site > Create site files - 46.84 ms - (96mb -> 100mb)
[PERF] Load site - 538.29 ms - (48mb -> 100mb)
[SUCCESS] Docusaurus website is running at: http://localhost:3000/
● Client ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ (83%) sealing chunk ids                                                                Panic occurred at runtime. Please file an issue on GitHub with the backtrace below: https://github.com/web-infra-dev/rspack/issues
Message:  byte index 2 is not a char boundary; it is inside '务' (bytes 0..3) of `务端-1-d-0-f-04c`
Location: crates/rspack_core/src/utils/compile_boolean_matcher.rs:325

Backtrace omitted.

Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
 ELIFECYCLE  Command failed.
[1]    97491 abort      DOCUSAURUS_PERF_LOGGER=true pnpm start

slorber commented 1 week ago

Thanks @ruibaby, that looks related to https://github.com/facebook/docusaurus/issues/10646

societymartingale commented 1 week ago

I enabled experimental_faster in the CI pipeline, and it reduced the build job duration by 3.5 minutes. This is for a Docusaurus site with 1100 markdown documents. I really appreciate the work that went into these optimizations!

ld-web commented 6 days ago

Thanks a lot @slorber, I upgraded today on a Docusaurus website with around 60 Markdown files including Mermaid charts, in Github Actions pipeline it went from average 1m30 build time to 12s ! Impressive ! 💯

mike-solomon commented 4 days ago

To add to the list of wins - with faster enabled on one of my sites with 4651 markdown files, the build job went from taking up to 15 minutes to build down to only ~7.5 minutes 🥳 . Thanks so much for the effort y'all put into optimizing this.

felipecrs commented 54 minutes ago

The build from an internal website here went from 4m18s to 1m8s. A great deal, thank you very much!

facebook / docusaurus