evanw / esbuild

An extremely fast bundler for the web
https://esbuild.github.io/
MIT License
37.98k stars 1.14k forks source link

[Bug?] Esbuild slow performance due to discarding dir cache when the plugin calls the built-in resolver #3795

Open pyrocat101 opened 3 months ago

pyrocat101 commented 3 months ago

Description

If the resolve plugin fallbacks to use the built-in resolver, a new resolver is created, causing dir cache to be rebuilt: https://github.com/evanw/esbuild/blob/67cbf87a4909d87a902ca8c3b69ab5330defab0a/pkg/api/api_impl.go#L2029-L2031

When there are a large number of files to be resolved, rebuilding these dir cache takes significant numbers of time, as is demonstrated in the repro below.

Repro

https://github.com/pyrocat101/esbuild-repro

This repro uses esbuild to bundle lodash-es, which has a couple hundred files to demonstrate the performance degration. The resolve plugin simply forwards the resolve request to the built-in esbuild resolver. When this plugin is enabled, it takes ~2.5s to bundle on my machine. But when the plugin is disabled, it takes ~0.2s.

I collected pprof cpu profile from esbuild, and it shows that most of time is spent in dirInfoUncached due to having to repopulate the resolver dir cache on every resolve call.

Impact

This issue affects use cases such as https://github.com/aspect-build/rules_esbuild, which uses a resolve plugin to make sure esbuild does not chase symlink out of the sandbox by post-processing the built-in resolver's result.

Potential Fix

Hoist the resolver.NewResolver call out of the resolve function definition to avoid creating it on every call. I built esbuild with the following patch applied and can confirm the performance has significantly improved:

diff --git a/pkg/api/api_impl.go b/pkg/api/api_impl.go
index 843f7f81..607d60cb 100644
--- a/pkg/api/api_impl.go
+++ b/pkg/api/api_impl.go
@@ -1994,11 +1994,17 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches

    var optionsForResolve *config.Options
    var plugins []config.Plugin
+   var res *resolver.Resolver

    // This is called after the build options have been validated
    finalizeBuildOptions = func(options *config.Options) {
        options.Plugins = plugins
        optionsForResolve = options
+
+       // Make a new resolver so it has its own log
+       log := logger.NewDeferLog(logger.DeferLogNoVerboseOrDebug, validateLogOverrides(initialOptions.LogOverride))
+       optionsClone := *optionsForResolve
+       res = resolver.NewResolver(config.BuildCall, fs, log, caches, &optionsClone)
    }

    for i, item := range clone {
@@ -2025,11 +2031,6 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
                return ResolveResult{Errors: []Message{{Text: "Must specify \"kind\" when calling \"resolve\""}}}
            }

-           // Make a new resolver so it has its own log
-           log := logger.NewDeferLog(logger.DeferLogNoVerboseOrDebug, validateLogOverrides(initialOptions.LogOverride))
-           optionsClone := *optionsForResolve
-           resolver := resolver.NewResolver(config.BuildCall, fs, log, caches, &optionsClone)
-
            // Make sure the resolve directory is an absolute path, which can fail
            absResolveDir := validatePath(log, fs, options.ResolveDir, "resolve directory")
            if log.HasErrors() {
@@ -2043,7 +2044,7 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
            kind := resolveKindToImportKind(options.Kind)
            resolveResult, _, _ := bundler.RunOnResolvePlugins(
                plugins,
-               resolver,
+               res,
                log,
                fs,
                &caches.FSCache,
@@ -2074,7 +2075,7 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
                if options.PluginName != "" {
                    pluginName = options.PluginName
                }
-               text, _, notes := bundler.ResolveFailureErrorTextSuggestionNotes(resolver, path, kind, pluginName, fs, absResolveDir, optionsForResolve.Platform, "", "")
+               text, _, notes := bundler.ResolveFailureErrorTextSuggestionNotes(res, path, kind, pluginName, fs, absResolveDir, optionsForResolve.Platform, "", "")
                result.Errors = append(result.Errors, convertMessagesToPublic(logger.Error, []logger.Msg{{
                    Data:  logger.MsgData{Text: text},
                    Notes: notes,