Tree-shacking with code splitting

[laurentgoudet]

For now the System-builder can:

• Build fast (scope hoisting) & small (tree-shaking) modules with buildStatic & Rollup
• Build less efficient System.register bundles enabling code splitting and lazy-loading

Webpack2 offers the best of both world, allowing code splitting while tree-shaking ES dependencies. Rollup also has plan to support code splitting (https://github.com/rollup/rollup/issues/372), as the current "externals" mechnism is very limited (single export per bundle).

Is there plans for the System-builder to offer a similar feature to Webpack2? My guess is that it could come with Rollup's support, but not sure of the details, especially as it looks more like an optimized version of bundle than the current behavior of buildStatic.

The reason I'm asking is that Angular 2 apps basically requires that: the whole NgModule architecture relies on tree-shaking to scale by eliminated the unused components of an NgModule (at least when using the AoT compiler), and the Router is centered around lazy-loading route bundles, which makes Webpack2 currently the only viable solution to use Angular 2 :/

[guybedford]

There is certainly a middle ground between build and bundle, and better tooling in this space is needed.

[ghempton]

This is the holy grail and a big missing link in all the different JS tooling system available. Vanilla System bundles support tree-shaking just as much as Webpack 2: an optimized bundle based off of specified entry points, but without scope hoisting. Without scope hoisting, Webpack 2 is still missing a lot of the performance benefits of Rollup (and is also the highest requested feature). Webpack is also still a little lacking when it comes to dynamic System.import calls. Rollup, on the other hand, does not support multiple bundles and, thus prevents code splitting. This becomes painful on the larger end of the app spectrum (where dynamic code loading can lead to better performance).

It is unfortunate that, even without Rollup, System.js does not provide more optimizations/features around dynamically loaded bundles/modules. This would be its killer feature over Webpack. Currently it is painful to have to deal with a very heavy-weight System.js loader in order to support dynamic code loading– but there aren't a lot of options.

[guybedford]

Anyone feel like working on a new module compiler?

[laurentgoudet]

System bundles support tree-shaking just as much as Webpack 2: an optimized bundle based off of specified entry points, but without scope hoisting.

That's actually a good point. My main issue with using bundle isn't really scope hoisting actually, but the fact that you then have to load the bundles configuration, which in my case (> 100 bundles) weights ~500kB and takes almost a second to be parsed :/.

I could be wrong but it seems to me that Webpack2 solves that by rewriting the imports within the production bundles, removing the need for the bundles configuration to be loaded. You lose the point of the module registry, but I'd argue that in a production workflow you probably don't need it (or it should contain your production "bundles", not the individual source modules).

[ghempton]

Wow 100 bundles seems like a lot. How many overall modules do you have?

From my limited experience, I found Webpack 2 surprisingly under-engineered for bundling and dynamic module loading. There do exist things like the Commons Chunk Plugin, but it also seems like it has some limitations. The way Webpack 2 rewrites System.import calls to magically infer bundles based on looking at the code itself is also extremely clunky coming from System.js.

I also agree that the system bundle config is incredibly verbose and a second to parse that seems unacceptable. It would be nice to be able to optimize that somehow with globs/wildcards.

[laurentgoudet]

Wow 100 bundles seems like a lot. How many overall modules do you have?

Around 1500 of our own, but well over 5000 including external libraries. It goes up pretty quickly as the JS community is increasingly adopting ES modules: for instance a small Angular 2 app with RxJS is >200 modules, only importing the .template() method from lodash-es is >50 modules.

The main reason we have so many bundles is because they are loaded on top of an old school server-side stack, so most pages will only load a couple of them, but the fact remains that the SystemJS bundles config doesn't scale well :/

The way Webpack 2 rewrites System.import calls to magically infer bundles based on looking at the code itself is also extremely clunky coming from System.js.

I think it depends where one see the future of JS bundling, or JS production workflow in general, especially now that the path for <script type="module"> and dynamic import() syntax is finally clear.

1/ Currently the System builder bundles mode is mostly about saving HTTP requests (through concatenation) & round trips (through the injected bundles config), and extra HTTP round trips can be saved for common bundles through depCache. But the basic assumption remains that the production tree/registry is the same than the source one, which is convenient for things like hot module replacement or dynamically loading any module.

One can imaging that long term, with ES6, HTTP/2.0, <script type="module"> and dynamic import() support, modules won't need to be bundled, and a "builder" output could be limited to generating modules dependency maps to avoid unnecessary network round trips. These maps won't scale either though, and handling them server side (e.g., h2 push + cache-digest) would require a custom assets server (expect if a standard comes up on that).

2/ The other scenario to me is that - unlike System builder buildStatic mode - a client-side Loader is used, however the production module tree is different that the source one. Imports are traced from user-specified entry points, and (optionally) through parsing dynamic import()s, to generate scope hoisted tree-shaken bundles: the Loader registry doesn't contain source modules anymore, which makes it impossible to load any random module on the fly, but the production bundles.

While it remains clunky, my understanding is that webpack2 enables / plans to enable that second workflow, as it allows for more optimized (smaller, faster to execute) code, at the cost of declaring all your possible dynamic import values at build time. Eventually they could even drop their custom client-side loader for the native import() (or a polyfill), but no matter what that scenario would require rewriting dynamic import() at build-time.

That scenario has drawbacks, e.g. heavily transformed code is harder to debug; loading source ES modules in dev would be nice, but to me the future of JS production workflow is likely to be more (advanced) transformations (unused object properties removal, binary intermediate format?) that less.

[ghempton]

This is a good synopsis of the two primary directions things are headed. It doesn't feel like System is inherently in conflict with scenario 2, but simply needs a more efficient way to configure bundles. Requiring each module to be mapped to a bundle is pretty limiting– perhaps if dependencies between entire bundles could be declared this could be made much more compact.