Minimize bundle size with tree-shaking

[pYr0x]

tldr; Update steal-tools build process to support tree shaking.

The RFC for this proposal was discussed on a recent live stream (1:16). It was also discussed on a previous live stream (38:52) and contributors meeting (1:05:29).

The Problem

When using 3rd party libraries you often are only using a portion of the functionality they provide. For example, if you are using a utility library that looks like:

hidash

export function intersection() {
  ...
}

export function flatten() {
  ...
}

But you are only using one of these utility functions like so:

import { flatten } from "hidash";

Even though you are only using one of the utility functions, your bundle will contain all of the code from this library's module.

The Solution

Tree shaking is a technique popularized by rollup that detects this unused code and "shakes" it away (and any code it depends on that is also not used elsewhere).

The other popular bundlers now support tree-shaking. We can piggyback on the libraries already created by others to support it as well.

• rollup is used internally by WebPack and the SystemJS Builder to treeshake ES modules
• common-shake is a utility that works with ASTs to tree-shake CommonJS code. As far as I know, no bundler currently uses this under the hood.

Tasks

• [ ] - Research how these libraries are used by bundlers to do treeshaking. I would think that they need access to the entire dependency graph, so we might need to plug this in as a transform that happens after transpile runs in steal-tools.
• [ ] - Create a couple of tests to show this working with ES and CommonJS modules.
• [ ] - Measure: I think presenting to the user how much wasted code was removed by using this technique will be a big benefit.

[matthewp]

For those unaware, the idea behind tree-shaking is that if you know which exports are used for a module, you can remove the code that is unused. For example:

foo.js

export function a(){}

export function b(){}

bar.js

import { a } from './foo';

Based on the fact that bar.js only uses a, we can remove the code for b (and any code that only b uses).

[matthewp]

Note that this is much harder for CommonJS because its exports are dynamic, not static like in ES modules. As your link shows, it might be possible with configuration. Steal could support some configuration so that you identify what exports are being used for a dependency, and based on that we would know what code could be removed. It would be quite a burden to maintain that though.

[pYr0x]

So you are thinking to only support es2015?

[matthewp]

Not necessarily. Some CommonJS modules are very static (most probably), it's only the truly dynamic ones that would be impossible to tree shake like:

var foo = require('./foo');

var fn = foo[Math.random()];

In the above we cannot statically determine what of foo's exports are being used, so we have to leave it completely alone. In some other cases we might be able to. But certainly CJS would be much harder, so we'd probably want to do ES modules first.

Regardless I think we can talk about it, and write up an RFC, that is high-level enough that it could be implemented for any format.

The high levels are we should be able to describe what imports a module uses and what exports it exports. Given this metadata we could do some stuff, probably in transpile to remove the unused parts.

[frank-dspeed]

@matthewp can't we simply https://github.com/google/traceur-compiler and then use es6 for tree shake as rollup does? with the lock file aka registry in rollup context? this would allow to exempt flag real dynamic objects and have also a method for detect if something realy is a dynamic object that gets exported?

[matthewp]

Update: we're thinking of making this a feature of StealJS 2.0. Both ES and CommonJS tree shaking.

[matthewp]

This has been written up as an RFC here: https://github.com/stealjs/rfcs/pull/20 . Please comment on the design there and I'll continue to iterate.

See also this repository which goes over a bit more detail in the design and why this algorithm/approach was chosen.

[matthewp]

This is in steal@pre now.