[FEATURE] Support rollup-style bundling

[dvoytenko]

Similar to the rollup this feature would combine all es6 modules with very minor modifications into a single module. In other words it'd flatten the modules. As with the rollup tool itself, this would make it easier to create export targets for sharing closure code between projects.

Consider the following set of modules:

File: parent.js

export class Parent {
  /**
   * @param {./utils/child.Child} child
   */
  constructor(child) {
     /** @const */
    this.child = child;
  }
}

File: utils/child.js

export class Child {
}

File: exports.js

import {Parent} from './src/parent.js';
import {Child} from './src/utils/child.js';

export {
  Parent,
  Child,
}

The closure_rollup tool with the exports.js as an input will produce the following single output module:

class Child {
}

class Parent {
  /**
   * @param {Child} child
   */
  constructor(child) {
     /** @const */
    this.child = child;
  }
}

export {
  Parent,
  Child,
}

In a nutshell, the closure_rollup --in exports.js --out out.js would do the following:

1. Read all modules, analyzes imports, exports and closure type annotations. Construct the dependency graph.
2. Order modules based on the dependency graph.
3. Remove all import and export statements from all modules.
4. Rewrite all types to be flat. E.g. {./utils/child.Child} -> {Child}.
5. Keep all other whitespace/comments as intact as possible.
6. Disambiguate duplicate names for top-level functions/classes.
7. Possibly do a very minor tree shaking: any top-level function, class or type not referenced anywhere could be removed.
8. Output all rewritten modules in order into a single module.
9. Complete the output module with the exports from the provided exports.js.

This kind of tool could allow a single well-used Closure project to create numerous export targets that would be easily shareable in the Closure form for other Closure projects to use.

[blickly]

Hmm, I think that we support something close to this when bundling WHITESPACE_ONLY mode with the --entry_point flag, although I'm not sure how well that combination is tested.

@jplaisted, do you know what we support for this setup?

[jplaisted]

No, this is pretty different. @dvoytenko is talking about a single module output. Our bundled output is not a module. Neither is our compiled code.

[jplaisted]

To put it another way: none of our outputs can be imported. The proposal is to output an ES module that can. As @dvoytenko stated, it flattens the input modules.

[blickly]

Hmm, I guess I was thrown off by bullet #3:

3. Remove all import and export statements from all modules.

[jplaisted]

Bullet 9 says to re-add some select exports at the end.

[ChadKillingsworth]

We would have to support ES_MODULES as an output target first - we don't do that currently.

[dvoytenko]

@blickly @jplaisted Yes, we'd like the ES6 output, just restricted to a subset of classes/functions/etc. The reasoning for this is this: locally in a project a module may want to do export function myUniqueUtility() {...} which is needed for project itself. However, as a inter-project export, myUniqueUtility is not needed. And on the contrary could be dangerous. Think of this as "module exports" vs "project exports".

Also, WHITESPACE_ONLY output is still very far from the original code (including surrogate goog.provide/require). As much as possible we'd like the output to match source, including whitespace and comments.

We currently maintain 4-5 Closure projects between which we would like to share such export targets. Currently we have to rely on the rollup and some really hacky type rewrites using regular expressions.

[concavelenz]

I expect you could create an ES6 module using and output wrapper today with appropriate "export statement".

[dvoytenko]

@concavelenz

I expect you could create an ES6 module using and output wrapper today with appropriate "export statement".

Could you pls clarify what you mean? Do you mean just appending export statement and mapping it to the recompiled/obfuscated names?

[ChadKillingsworth]

Like this:

Custom Extern:

const moduleExports = {};

Source

import foo from './source2.js';

moduleExports['foo'] = foo;

Compilation Command

java -jar compiler.jar --js source1 --js source2 --language_out ECMASCRIPT_2015
    --externs custom-extern.js
    --output_wrapper="export const foo = (function(moduleExports) {%output%;return moduleExports['foo'];})({});"

[dvoytenko]

@ChadKillingsworth Thanks! But this still makes the overall code completely unrecognizable w.r.t. to the source code: comments are lost, types are lost, etc. We'd really like this to be a proper Closure code with flattened types.

[ChadKillingsworth]

That seems very much outside of the compiler's scope then.

[dvoytenko]

@ChadKillingsworth certainly you can reject this ask. However, this kind of bundling for code sharing would seem very beneficial for Closure-to-Closure project dependencies. And, of course, only Closure knows all the right type rules to change the references correctly. E.g. @param {./module1.Class1} -> @param {Class1}.

[ChadKillingsworth]

Bundling modules is definitely doable. Preserving type annotations is a big "maybe" (it's there in the compiler, but it's not perfect).

But preserving comments in general or expecting the output code to be perfectly human readable - that's where things become problematic. That's never been a goal of the compiler.

[dvoytenko]

@ChadKillingsworth Certainly. And as I mentioned above, we're for now doing ok by executing rollup and regex-ing types. There are occasional errors and stuff though.