Treeshaking when using `esbuild-wasm` on the browser isn't as efficient as when running locally

[okikio]

In order to supprt npm packages I'm basically using a plugin to auto fetch the packages and their files from https://unpkg.com, I'm manually fetching the package.json as well and setting up the resolves all manually, but this method seems to cause issues with esbuilds treeshaking system.

e.g. https://github.com/okikio/bundlejs/issues/51 https://bundlejs.com/?q=date-fns&treeshake=%5B%7B+isAfter+%7D%5D

^ For this specific bundle of date-fns only { isAfter } is exported, but the final resulting bundle for some reason brings in the parser, I can't find the reason why the parser is brought in

[okikio]

I've found that if I use esbuild as an import traversal system to fetch all the packages and files required, and then run esbuild again the treeshake then works, is there a better approach to get esbuild treeshaking to work better?

[evanw]

Can you be specific about what issues it causes? Why is tree shaking happening in some cases but not others? Sorry, I don't understand why this is happening from what you've written.

[okikio]

@evanw Hey there, sorry if my previous explanation was a bit muddled. Let's dive back into it.

I'm seeing some intriguing behaviors with esbuild and I'm hoping you can help shed some light on what's happening.

Fetch and Go

For starters, I'm using a plugin to resolve imports. With an import like import react from "react", I'm fetching it directly from https://unpkg.com and then passing it off to the plugin loader. This gets me the bundle I need, but tree shaking seems a bit off.

I've put this method side-by-side with a plain vanilla esbuild-wasm run and noticed that esbuild tree shakes more effectively when plugins aren't involved in loading. However, when a plugin gets in on the action—fetching packages externally—tree shaking isn't as robust.

Fetch, Store, Reuse

Taking it a step further, I've started caching loaded files in a temp virtual file system and using a plugin to load from there.

Here's the workflow: I fire up esbuild, giving it marching orders to only focus on loading the external files into the virtual file system on the first run—kind of like a file scout.

After that initial recon run, I ditch esbuild's results, then run esbuild again. But this time, esbuild is directed to only use the virtual file system plugin for package resolution. Lo and behold, this gives me a properly tree-shaken bundle.

It's puzzling as to why this is happening. I'm noticing that when all the files and modules are already hanging out in memory, esbuild fetches them lickety-split. This makes me wonder if there might be a race condition causing the tree shaking to stumble.

I hope that makes my situation a bit clearer. If you have any insights into this, I'd be super grateful to hear them. Thanks in advance for your help! Feel free to ping me if you've got more questions.

[evanw]

Is there any difference in the metafile for both builds? For example, is one build using that package's main field while another build is using that package's module field? That would explain a difference in tree shaking because tree shaking only works with ESM, not with CommonJS. What you described (downloading the files ahead of time vs. on the fly) sounds like it should should be providing the exact same inputs to esbuild, which should then result in the exact same outputs. If the inputs to esbuild are different in these two cases, then it sounds like it'd be due to some problem with the plugin you wrote, which you should be able to fix for yourself. In any case, providing a self-contained way to reproduce the problem you are describing would be helpful here.

[okikio]

I'll try to create a self-contained reproduction

[okikio]

@evanw Sorry for the delay it took me a couple days to get a clean repro https://github.com/okikio/esbuild-wasm-treeshake-repro

Note: the approach taken in the repro doesn't cover edge cases, to improve readability

The result of using esbuild without plugins on nodejs to bundle and treeshake export { isAfter } from "date-fns"; is 1.5Kb treeshaken and bundled, and 99.8Kb when using the plugins, with treeshaking and bundling enabled

[hyrious]

Well I found something in the logLevel: 'verbose' log:

Marking this file as having no side effects due to "/node_modules/date-fns/esm/add/package.json"

That is to say when using plugins, esbuild won't know lines below has no side effects becasue it won't check package.json without FS access.

// date-fns@2.30.0/package/esm/index.js:2
export { default as add } from "./add/index.js";
// and other hundreds of lines ...

---

But, if using the same trick in https://esbuild.github.io/try to simulate a real FS, then maybe you can get the ideal result..

[okikio]

OH!! OH my gosh, that explains a lot, what heuristics from the package.json does esbuild use to determine if something has sideEffects

[okikio]

@hyrious @evanw Is there a way to let esbuild-wasm know of a package.json file to use from a plugin?

I tried replicating the problem on https://hyrious.me/esbuild-repl/?version=0.18.0&b=e%00entry.js%00export+*+from+%22date-fns%22&i=date-fns%402.30.0&i=%40babel%2Fruntime%407.22.5&i=regenerator-runtime%400.13.11&o=--bundle+--format%3Desm but it also runs into the same sideEffects treeshaking issue, I think I have a solution on https://github.com/okikio/esbuild-wasm-treeshake-repro but it's messy and not ideal

[hyrious]

AFAIK there's no way to let a plugin provide such info. That's why https://esbuild.github.io/try/ needs the hacking way to provide FS in browser.

[okikio]

😭 :sad esbuild noises:

[jakebailey]

Technically, you can use WASI instead to run the CLI version in the browser, using a full VFS: https://github.com/jakebailey/esbuild-playground/blob/main/src/esbuild/wasiWorker.ts

But, you won't get the JS API, only the CLI.

[okikio]

That unfortunately wouldn't support npm packages

[jakebailey]

It's a VFS; you can mount whatever you want into it. I'm not sure how you're getting packages into esbuild-wasm in the browser as it is now; plugins?

[okikio]

Yep, plugins

[okikio]

Thanks for all the help I've finally figured out a solution to this problem 😅, I'm manually grabbing the side-effects property from the package.json and manually passing it through using the vfs plugin