Discussion: Fix Plugin Ordering

[hzoo]

Copying from https://github.com/babel/notes/pull/19, many other issues linked there

Issue

The order that you specify plugins/presets in a config matters in Babel. User's shouldn't have to change their config to make it compile correctly (or get a weird error message about how a plugin isn't enabled when it's just in the "wrong" order).

From the https://github.com/loganfsmyth/babel-plugin-transform-decorators-legacy repo

Simpler example: if you specify a plugin only modifies ES2015 classes and you make that run after the plugin that transforms classes, then it won't run at all because at that point the classes will have been transformed into a function already.

{ plugins: ["modify-es2015-classes", "transform-es2015-classes"] } // works { plugins: ["transform-es2015-classes", "modify-es2015-classes"] } // fails

How can we make Babel automatically know how to order the plugins for the user, or at least give a clearer error message? Or how can we let the user opt-in to a specific ordering?

Duplicate plugins in a config

What if a user includes "es2015", "env", "latest", and "transform-arrow-functions"? That includes the arrow function transform 4 times! It would be nice to have a nice error message to explain that there's duplicate functionality or that you have a plugin that isn't already on.

Possible Solution: Capabilities Discovery

• Plugins provide a function to state the capabilities it provides and which it needs.
• This includes both syntax and built-ins
• This is important to fix plugin ordering automatically in Babel itself. If the classes plugin transpiles classes and decorators run over classes, then with this information Babel will know to make decorators run first.

This could mean exposing a method in plugins called hasCapability('capability-here')

// minify-plugin.js
// return true or false based on if the target environment natively supports arrows
// or if the plugin that compiles arrow functions is enabled
if (hasCapability('es2015-arrow-function') {
  // output an arrow function
} else {
  // output a function expression
}

Other ideas

• Allow plugins to have multiple priorities (and a lower priority can error out to signal that a plugin should move before or after)
• Explicit ordering of before/after by users (sounds like passPerPreset) (#3793, #3281).

Testing

A test can be to randomly sort the plugins in preset-env/es2015/presets and check if the output is the same. Also tests for specific issues we know about (decorators and class properties)

Other Benefits

• Users shouldn't need to care about what order plugins are specified
• This would also provide better error messages: "your native environment doesn't support this syntax and you haven't added the correct plugin to transform it: here are some options and how to install"
• Minify can also take advantage of this by having multiple outputs based on the target environment (output arrow functions (ES6) if available)

[mhelvens]

I've noticed the same problems with the way babel currently does plugin ordering. I have some ideas which, by coincidence, I posted in issue #3793 just earlier today. I'll repeat them here (with minor changes for context).

Problem

I agree that babel users shouldn't have to understand compilation phases / stages or the subtleties of presets in order to use babel to its fullest. And plugin run-order should ideally not depend at all on the textual order of the plugins and presets arrays in .babelrc. Ideally those arrays should just represent a set of plugins and a set of presets (which are, in turn, sets of plugins and presets).

Proposal

My proposal is based on having the set of plugins be a partially ordered set (or, if you like, a directed acyclic graph), and having the order be specified as "before" / "after" rules that refer to other plugins (or semantic concepts; see below). The simplest example of this would be that pluginThatNeedsFlowTypes would specify that it wants to run before flowStripTypesPlugin by specifying this either in its own .babelrc or as a property / method inside the plugin itself:

{ // pluginThatNeedsFlowTypes
    "before": ["flowStripTypesPlugin"]
}

Or, if the plugin author wasn't aware of the existence of the other plugin, the user might specify this order in their own local .babelrc as a last resort:

{
    "presets": ["es2015"],
    "plugins": ["flowStripTypesPlugin", "pluginThatNeedsFlowTypes"]
    "order": [ // set of pairs that specify partial order
        ["pluginThatNeedsFlowTypes", "flowStripTypesPlugin"]
    ]
}

The resulting partial order has to be respected by babel. Naively, it should run them in some topological order, though optimization techniques can be used if they don't cause the output to change. For example, plugins that are unordered in the graph can probably be run in the same traversal with no problems.

The advantages of this approach include:

• Dependencies are explicit;
• Plugin authors can change their own ordering / dependencies without forcing users to update their configuration;
• Cycles in the graph can be easily detected (presumably prompting plugin authors to collaborate on a solution);
• Normal users don't need to know any of this.

Scaling to semantic concepts

The obvious problem with the naive version of the proposal described above is that every plugin would have to know about every other plugin with which it might conflict, and that's not scalable. To address this problem, we can allow arbitrarily named nodes in the graph that represent semantic concepts, to be used as common points of reference for plugins. For example, plugin authors would do something like this:

{ // pluginThatNeedsFlowTypes
    "before": ["(flow-types-exist)"]
}

{ // flowStripTypesPlugin
    "after": ["(flow-types-exist)"],
    "before": ["(flow-types-do-not-exist)"]
}

{ // pluginThatDoesNotUnderstandFlowTypes
    "after": ["(flow-types-do-not-exist)"]
}

In these examples, we assume that the plugin authors are all aware of the (flow-types-exist) and (flow-types-do-not-exist) concepts. If they properly order their own plugin relative to those concepts, they don't have to be aware of each other specifically.

As for the concepts themselves, I guess babel would have to enforce some naming convention or other, and here I've assumed that their names have to be enclosed by parentheses.

I'm ambivalent on whether or not these concepts should need to be registered or declared somewhere. On the one hand, we could allow unrestricted use of them, and the community would probably converge on a common set. On the other hand, if babel was aware of them, it could give better feedback. Perhaps a partially ordered set of related concepts could be released as package (which I'll call a scaffolding), and plugins could import it to order themselves relative to it.

There are undoubtedly ways in which this proposal can still be improved, but I think I've made the general idea clear. I'd be happy to elaborate if needed. If you don't think this idea will fly, fair enough. It is a rather radical departure from the status quo (although I believe it can be made fully backwards compatible for users).

(PS: This proposal is loosely based on delta modeling, the topic of my PhD thesis. I've written delta-modeling libraries for LaTeX and JavaScript).

[TrySound]

I found that "env" section presets are run before plugins, which is not quite clean according documentation, so I can't just have simple config with class properties and es2015 preset

{
  "plugins": [
    ["transform-class-properties", { "loose": true }]
  ],
  "env": {
    "build": {
      "presets": [
        ["es2015", { "loose": true }]
      ]
    }
  }
}

[hzoo]

That is more of an issue with the "env" key which we are deprecating in favor of using .babelrc.js so you can specify it yourself.

[TrySound]

@hzoo Oh, that's nice! However package.json compact way will be gone then.

[AshleyScirra]

Another use case of ordering is for minification. My property mangling plugin has conflicts with other plugins and currently must be run first. There are two reasons for this:

1. As per Closure Compiler's approach, the property mangler only mangles o.property syntax, not o["property"] syntax. This provides a convenient escape hatch for things like external API calls. However more than one babili plugin does a minify transform to convert o["property"] -> o.property. This breaks this approach since it causes everything to get mangled, so ordering must be enforced to prevent other plugins running first.

2. Avoiding name collisions with the local name mangler is also very complicated, since the property mangler also mangles global-level names. Here's an example of the problem:

window.foo = 1;
function test(bar)
{
  console.log(foo + bar);
}

If this mangles like this, it's broken:

window.a = 1;
function test(a)
{
    console.log(a + a); // oops: does not use global variable
}

So the property mangler and local mangler have to co-operate to ensure they do not pick colliding names. This becomes extremely thorny when you process multiple script files, since you can't pick a global property name until you've seen all future scripts and know what local names will be used across all of them. Doing this would require even further rearchitecting of Babel, which I don't think is reasonable. Instead I opted for a much simpler solution: the local mangler already makes sure it doesn't collide with global names, so as long as the property mangler runs first, the local mangler does the work of ensuring none of the names it chooses collides with anything. This is a nice solution especially since only one plugin handles collision avoidance rather than both having to solve that, but again depends on ordering: the property mangler must run first.

Optional ordering

It's probably also worth mentioning optional orderings. My experimental string literal minifying plugin, although probably not useful in practice, demonstrates one interesting problem: it has to choose the shortest non-colliding local variable names possible. This duplicates the work of the local name mangler. So what it does instead is generate relatively safe names like const _1 = "a long string", ..., _100 = "another long string". Then as long as it's run first, the local name mangler later does the job of picking even shorter names that don't collide with anything else. The plugin never makes the uncompressed script longer, so it doesn't really matter if it runs before or after the local mangler, but it's better if it does run first. So it's an optional dependency.

Common sub-expression elimination is probably a similar case: you have to pick the shortest possible non-colliding temporary variable names. (In fact this kind of string deduplication is a very limited case of that.)

Problems with other ideas

I worry that any means of forcing an automatic ordering will eventually fail. Sooner or later you will have "A must be before B which must be before C which must be before A". This is unresolvable, and you would probably have to entirely remove one of the plugins.

Ordering by specific plugin names seems brittle, and I also suspect semantic ordering is also too brittle. Programs can do arbitrary things, and coming up with useful tags to describe them also sounds like it won't scale well. For example what tag would the string minifier use? "generates-local-variables-which-may-optionally-be-later-renamed-as-long-as-its-shorter"? Are third-party developers really going to look up semantics like those and ensure they tag their plugins correctly?

Proposal: named passes

In my view the best solution is a variant on the priority idea. If plugins are simply sorted by a priority number, I doubt anyone will respect other plugin's priorities, and I suspect a lot of plugins will simply choose first or last, causing the same problem at either end.

Instead I think babel's processing should be split in to separate named passes, each with its own specific purpose, something like this:

0. Information pass: a read-only pass to collect information, like the set of identifiers used, character frequencies, etc. Plugins know this is the original source that has not been modified by anything else yet.
1. Generation pass: a write-only pass where only new nodes can be inserted: existing ones cannot yet be modified. This means things like polyfills can be added, or new local variables inserted, with the guarantee they will be transformed by later passes. This is good for minification, since it guarantees anything added will be fully minified, rather than risking adding something when minification has happened.
2. Pre-mangling pass: a pass where anything can be modified except existing identifier names.
3. Name mangling pass: a pass intended for the property and local manglers, where anything can be changed including identifier names. This would probably need to be two passes in practice to guarantee the property mangler runs before the local mangler.
4. Post-mangling pass: one more pass where anything can be changed, in case anything needs to run after mangling.

Limitations could be programmatically enforced, e.g. throwing an exception if changing anything in the information pass. This would provide useful guarantees to plugin authors.

Obviously for performance if no plugins use a pass, it's not run. There are actually some interesting optimisation and modularity opportunities with this, e.g. making a character frequency counter its own plugin running in the information pass, and sharing the information with both the property and local name manglers, ensuring they both use one implementation and the same information.

This design is minification-centric (it's what I've mainly worked on) but it does solve a lot of problems. For example:

• property mangling runs in the mangling pass; o["property"] -> o.property transforms happen in the post-mangling pass; now they don't conflict
• using separate passes for property vs. local mangling ensures they run in the correct order (this feels too specific a use case, maybe a secondary ordering mechanism could be used instead)
• string minification (or full common sub-expression elimination) runs in the pre-mangling pass, ensuring it applies to any inserted polyfills, and also that the local name mangler then picks optimal names in the mangling pass

It still doesn't solve everything. Plugin authors must still collaborate to pick the right passes, e.g. taking in to account mangling. Also I don't know enough about the transform-decorators-legacy vs. transform-class-properties conflict to know if this will solve it. Can anyone comment on why that conflict happens? Perhaps the passes can be adapted to solve that too. Alternatively a secondary priority system within each pass could be used (not perfect but gives extra flexibility), or just allow manual ordering as a last-ditch compatibility option.

I wouldn't be surprised if this ended up being extended to something like ~10 passes to cover all the use cases in babel, I don't think anyone knows all the ordering issues that currently exist or will come up in future.

Avoiding running something like an arrow function multiple times is a different problem really. Babel could provide a flag to make sure plugins are only run once, or allow plugins to store state across all passes so they can have a "has run" flag.

I believe using named passes would solve many ordering problems (probably not all, but many cases), provide more predictability and structure to the many varied transforms babel plugins make, help guide plugin developers in to doing work at the correct time, and allow better modularity and information sharing.

[mhelvens]

@AshleyScirra:

I also suspect semantic ordering is also too brittle. Programs can do arbitrary things, and coming up with useful tags to describe them also sounds like it won't scale well.

I don't really understand this argument, given your own proposed solution. Your named passes are just a set of semantic concepts in a strict linear order. That is, they can be fully encoded in the system I proposed.

For example, your chain of name-mangling passes is probably a sensible scaffolding to release for minification-related plugins. But there will be other such chains that are unrelated to minification (either partly or completely), and should not be forced into a linear order with it. The more plugins that are able to remain unordered, the more freedom Babel will have to optimize the run.

Are third-party developers really going to look up semantics like those and ensure they tag their plugins correctly?

It's not much different from having to look up and choose the correct pass, except that developers will only have to choose semantic concepts that are meaningful to them, e.g., "All I want is for my plugin to run before Flow types are removed. I don't want to know if that's before or after local name mangling."

All that said, when it comes down to it, I think we mostly agree on what's needed, and you've convinced me on several points, which now follow:

Ordering by specific plugin names seems brittle

Agreed. I suspect most ordering would be by reference to semantic concepts. Though ordering relative to specific packages is still a powerful feature to have, and there's no reason not to offer it.

coming up with useful tags to describe them also sounds like it won't scale well

Yes, semantic concepts should be declared (in scaffolding packages) before they can be used, and the Babel team should release a set of common ones. Let's not allow free-form strings to be used. That way chaos lies.

Sooner or later you will have "A must be before B which must be before C which must be before A". This is unresolvable, and you would probably have to entirely remove one of the plugins.

Yes, it's possible for cycles to appear when developers make mistakes or don't communicate with each other properly. The most likely sort of scenario where this might happen is this:

• (names-mangled) --> plugin-A --> (flow-types-removed)
• (flow-types-removed) --> plugin-B --> (names-mangled)

If / when this happens, it will often be a sign that one or both plugins are doing multiple jobs in a single pass (premature optimization?), and should be split up into separate plugins or separate (virtual) passes:

• plugin-A:
  • plugin-A-pass-1 --> (flow-types-removed)
  • (names-mangled) --> plugin-A-pass-2
  • plugin-A-pass-1 --> plugin-A-pass-2
• (flow-types-removed) --> plugin-B --> (names-mangled)

This brings up another point you made:

There are actually some interesting optimisation and modularity opportunities with this, e.g. making a character frequency counter its own plugin running in the information pass, and sharing the information with both the property and local name manglers, ensuring they both use one implementation and the same information.

Yep!

[AshleyScirra]

I also suspect semantic ordering is also too brittle. Programs can do arbitrary things, and coming up with useful tags to describe them also sounds like it won't scale well.

I don't really understand this argument, given your own proposed solution. Your named passes are just a set of semantic concepts in a strict linear order. That is, they can be fully encoded in the system I proposed.

Well, they are related, it's just changing some of the practical aspects. I interpreted your proposal as allowing any plugin to define any semantic concept it likes, and then any other plugin can order based on any of those semantic concepts. In theory it solves everything, but I don't think this will scale well in practice: it'll end up creating effectively a database of all sorts of different semantic concepts, and for a casual plugin dev to work out the ordering of their plugin in possibly hundreds of concepts probably means they'll just ignore it or copy someone else's. In practice people will get things wrong and then you might end up with unresolvable requirements, possibly spanning several different author's plugins, and then trying to figure out whose plugin should be changed would be difficult (and probably contentious).

The main difference with passes is:

• there aren't many
• all plugins have to use the existing set, third-party plugins can't affect the passes
• each pass has a clear high-level purpose, rather than trying to micro-manage individual concepts

The aim is to make it more practical to manage, rather than aiming for theoretical perfection. I think there'll still be awkward cases with using a pass-based architecture, but at least it puts an overall high-level organisation to compilation.

[mhelvens]

I interpreted your proposal as allowing any plugin to define any semantic concept it likes, and then any other plugin can order based on any of those semantic concepts.

Yes, but it won't be as chaotic as I think you're thinking.

First, let's separate some concerns here. Rather than pit one full proposal against another, let's acknowledge the similarities (using the terminology of my 'more expressive' framework):

• Both propose ordered sets of semantic concepts into which plugin runs are inserted.
• Both say that plugin authors, not users, should decide where their plugin is inserted.
• Both allow plugins to be split up into disconnected runs that can pass information forward.

If you haven't read to the end of my previous comment, I suggest you do now. I talk more about areas of agreement there. And then, let's argue about the two main differences (to the extent that they are orthogonal):

1. Partial order vs total order.
2. Free vs restricted creation of semantic concepts.

Partial order vs Total order

You propose a total order for the semantic concepts, and for placement of plugins to be between two subsequent concepts (= inside a single phase). Plugins in the same phase are not mutually ordered, but this can be done manually if required.

You haven't raised objections to the "partial order" nature of my proposal. (Your three differences all relate to "free vs restricted creation".) So perhaps you already agree with it. But let me make a quick case for them.

You get closer to a partial order by loosening the above-mentioned restrictions. One simple step we might take, for instance, is to allow plugin authors to choose a range of phases in which their plugin could run, rather than a specific single phase. For example, if a plugin only requires ES6 classes to still be present, why force them to make a choice between the 7 phases in which this is the case? They don't know and shouldn't have to care. Just let them say: "This plugin must run before ES6 classes are translated." or "... before the ES6-to-ES5 phase" or something. This is simpler for the plugin author, and Babel can run that plugin wherever is most efficient. Win-win.

The next step is to allow any two semantic concepts not to be ordered either. "Name mangling" and "Flow type removal" are two independent operations, and their order doesn't matter. Not having to specify that order can only be advantageous for all involved.

Continuing this line of reasoning, the logical conclusion is to just allow any partial order. Partial orders are the most natural structure for plugin systems, and they're easy to analyze.

Free vs restricted creation of semantic concepts

there aren't many [passes] all plugins have to use the existing set, third-party plugins can't affect the passes

Please note that the specific passes you propose are precisely the ones you would need for your own plugins. You concede that there will probably be a need for other passes, and estimate the total number at ~10. All I can say is that I don't think you've really thought that number through.

each pass has a clear high-level purpose, rather than trying to micro-manage individual concepts

Your list includes three whole passes related to name mangling. Others may well see that as low-level micro-management. But you could probably defend the need for those passes, because you have experience and expertise in that area. By the same token, other plugin authors may see the need for specific passes in their own area. Why prevent them from scaffolding those passes?

I don't think anyone knows all the ordering issues that currently exist or will come up in future.

Exactly.

Now, I do see that free creation of semantic concepts could get out of hand if it wasn't assisted by good documentation and tools. Mostly, I propose that a register be created and automatically maintained from npm. The register knows about existing semantic concepts and their ordering, and could display more prominently the ones that are being downloaded the most. Plugin authors should consult that list before creating their own concepts. They'll see the advantage of that.

(I'm actually imagining a registry page with a d3-based graph of concepts and plugins, filtering by various criteria. It'd be super-useful, but by no means a requirement in the near term.)

Additional stuff

they'll just ignore it or copy someone else's. In practice people will get things wrong and then you might end up with unresolvable requirements

Unresolvable requirements can only occur when a cycle is created that contains a plugin that is being used. In other words, when there are too many arrows in the graph. If casual plugin authors ignore the system or copy someone else's ordering, that could never create a new cycle.

Worst case scenario is that they don't restrict their plugin enough, so that it is run in the wrong position. That could happen with any system we might propose, if we assume lazy plugin authors.

The aim is to make it more practical to manage, rather than aiming for theoretical perfection

Oh, same for me. I'm convinced my proposal would make the system a great deal more practical for Babel users and plugin authors, admittedly at the cost of additional complexity for Babel developers. But I do think you're overestimating that complexity.

[hzoo]

This doesn't have to solve every possible plugin or combination of plugins, certainly not initially. Our original goal is just so that all transform plugins/presets at least in babel/babel (either in JavaScript or experimental) don't require any user to have to order them in a specific way. This is the 99% usecase.

Thus the "semantic" concept we had in mind is just what syntax (arrow functions, classes, other visitor nodes) each plugin handles/outputs.

[mhelvens]

@hzoo: Sounds sensible.

I would only advise (at the risk of stating the obvious) to err on the side of flexibility, and keeping your options open. For example, you may only have syntax-based 'concepts' now, but don't couple 'concepts' too tightly to the syntax system. Make them a generic feature and just start out with syntax-related concept names.

[hzoo]

If you need to make a specific ordering, you can already just put that plugin first or just create a preset with it's own pass.

My proposal is to use syntax as the concept to know that certain plugins could run before the transform plugin is run. It would just do an in order sort such that it moves the relevant plugin in the correct position by going before the transform plugin.

[hzoo]

Regarding the running of plugin ordering (topological), sounds like what @seansfkelley did in https://github.com/lerna/lerna/pull/358 for lerna to sort packages? Any thoughts Sean?

[hzoo]

Made https://github.com/babel/babel/issues/5735 for the immediate use case - we can talk about how to expand to passes or minification too

[hzoo]

Going to move to another issue https://github.com/babel/babel/issues/5854