Use HTML feature opt-in for language level compatibility

[jonrimmer]

Since the only problem with proposals like "flatten" (and others) is in the web runtime, why is this a problem that needs to be solved at the standard-library level? I agree that web-compatibility has to be considered, but wouldn't the more appropriate place to solve it be in the browser?

There is already work underway at the W3C to support explicit feature opt-in (and opt-out). Couldn't there be a feature-policy for ES compatibility as well, either through the kind of HTTP-header mechanism described in the linked document, or a more traditional, but still preload-scannable pragma like

<meta name="script-compat" content="es2019">?

This would have the advantage of solving web compatibility more generally, rather than requiring a debate and bespoke solution every time an ancient library causes problems. It would also make native prototype extension a potentially useful part of the language, instead of a compatibility footgun.

[bakkot]

After long experience the committee has generally come down quite firmly against new modes. There's summaries of this position in various places (e.g. here), but briefly: they have a lot of edge cases (how do they interact with realms? what about eval? can they be feature-detected? what happens if two browsers ship different things under the same "version"?), drastically increase maintenance burden on browser vendors, website maintainers, and the authors of test suites, and overall do not seem worth the cost.

[jonrimmer]

I think any topic should be open to review, regardless of historical conclusions. Technology is not a field where any decision should be set in stone, particularly when there will always be new participants and new opinions coming to the table. Furthermore, given this is supposed to be a forum for discussion, instantly closing down suggestions on the basis of your preexisting beliefs (whether they are right or wrong) appears very dismissive and unwelcoming. It suggests you are not interested in even countenancing the views of "outsiders" who do not subscribe to the attitudes you have institutionally decided are the correct ones.

[jonrimmer]

As to your specific objections, the discussion of 'use strict' in the document you is linked interesting. I think 'use strict' was awful, but that its problems were not unavoidable. Rather than taking it as a reason to abandon explicit versioning entirely, it should instead serve as a lesson for how to do it better:

1. It (ab)used strings as an in-band pragma, thus feeling weird or hacky to some developers.
2. It required declaration in every single script file, thus creating a frustrating burden on developers..
3. An opaque choice of words that suggests little about its actual purpose.
4. Most importantly, it doesn't actually give you anything as a developer. It just takes stuff away.

Given the above, it is any surprise that 'use strict' is unloved and often unadopted? 'Use strict' is a perfect example of something driven by what seems sensible to language designers, without proper consideration for developer ergonomics. It was treated as an opportunity to clear up a few warts, and nothing more.

But we can solve each of the above problems:

1. Use a proper pragma declaration mechanism, so it doesn't feel hacky.
2. Require a single declaration, achievable via the meta tag or HTTP header mechanisms I suggested.
3. Have a sensible and non-single-use naming scheme. Explicit opt-in to a particular ES version is a lot clearer than saying you want to "use" some notion of strictness.
4. Actually provide features as part of it. If developers have to opt-in to get features like flatMap and flatten, then they're going to do it.

None of this is unprecedented, even on the web. Beyond the W3C proposal I linked previously, Google and others have been using feature availability as a way to drive adoption of HTTPS. Like it or not, this is the way the web is evolving, because it's the only way to move forward in a sustainable way. TC39 adopting a philosophical taboo against it for historical reasons is not helpful.

I am also not sure what you mean by a "drastically" increased maintenance burden on website maintainers and test suite authors? I can certainly see that browser vendors will have to do some added work to support this. But with a single major ES version per year (many of which would probably not require a new compatibility mode), this seems manageable.

Website maintainers would have no more or less of a burden. They would not be required to opt-in to newer versions, any more than the authors of ancient websites are required to update their sites now. The only difference would be that developers working now would be able to explicitly declare the version of ES their code is compatible with.

Test suite authors would simply write tests against particular features. If run in a browser runtime with the pragma for the new version, those tests would pass. If run in one with a pragma for the old version, they would fail. Where is the maintenance burden?

You other points simply seem to be technical questions of design and implementation. I can't say what the interaction with realms and eval would be, but I can't see how they are insurmountable challenges to the entire concept of versioning. At the very least, there is prior art, as versioning is at the heart of how technology manages to progress while retaining backwards compatibility. For the group charged with stewarding the ES language forward to be so dead-set against even considering it seems perverse.

[bakkot]

Sure, any decision is in principle open to review, and you certainly needn't trust my opinions! I can go over the process for getting this in front of the rest of the committee if you like; I can even bring it up at this next meeting myself. (It wouldn't be in this repo, though.) Just, I don't think many of us feel like that decision was made incorrectly, and as such unless there's dramatic new evidence it's extremely unlikely we're going to change it at this juncture. We can't review every decision at every point in time.

[jonrimmer]

Fair enough. It's just that instantly closing issues feels rather like slamming a door in someone's face. Maybe your policy could be to leave them open for a day or two at least, to see if they garner any interest? It's not like this repo is so overrun with activity (the last day or two notwithstanding).

I actually would appreciate it if you brought it up. In particular, I'd be interested in hearing the committee's opinions on my points above. Or, perhaps more succinctly:

1. Is the committee certain that it is not allowing negative experience with 'use strict' and vendor-specific extensions to colour its opinion of all versioning pragma options, in particular those with better developer ergonomics than 'use strict'?

2. Is the committee certain that the drawbacks of a version pragma have been adequately weighed against the opportunity cost of not being able to extend and improve the language, not being able to correct their mistakes, the burden on library authors to avoid any potential conflict with future ES versions, and the seemingly negative opinion of a section of the developer constituency with the contortions necessary to satisfy the "One JavaScript" philosophy?

3. Has the committee properly considered the changed environment with respect to browser auto-updating , inter-vendor compatibility, and stable ES version cadence, with respect to the problems of incompatibility between implementations?

   4. Has the committee properly considered the more general trend towards opt-in pragmas and modes in the web platform at large? In particular, I'd be interested in how the committee's representatives from Google square the TC39's position with their employer's otherwise enthusiastic embrace of technologies like AMP that seem to be very deliberately breaking with the past via opt-in mechanisms.

[ljharb]

@jonrimmer Just to set your expectations: it's come up multiple times in committee and the overwhelming response is always "no new modes, ever". The opportunity cost of not being able to extend a prototype's API with a single name, given the entirety of the english language being available, is generally deemed as insignificant compared to the complexity of a mode or a version pragma.

[jonrimmer]

@ljharb I don't understand why you and @bakkot keeping saying "mode" when what I am proposing is not a mode. It is not an in-band pragma. It is not a per-script or per-module pragma. It is not 'use stricter mode'. It is a single out-of-band version switch that would apply to each and every script executing within the scope of a particular runtime execution context (e.g. a browser page load).

As such, there would be no multiple paths of behavior defined in the spec. There would be not need for complex interoperability definitions between code running in different modes and scopes. There would be a single artifact: the ES2019 specification, describing how ES2019 code runs. It would be no different than the ES2017 spec, or the ES2016 spec, or the ES2015 spec, etc. Since you managed to successfully publish those artifacts, I have faith in your ability to do so again.

The only difference would be that this version would be free to choose sensible API names, without reference to the historical decisions of library authors, and would also be free to make syntactical and behavioural changes, and fix bugs, even if they resulted in some degree of backwards incompatibility. And the spec would include a note indicating the if, and only if, runtimes wished to retain full backwards compatibility with earlier ES versions, then they should provide authors with some mechanism, either opt-in or opt-out, by which code could choose which version of ES it wished to execute with.

Given that, as stated, browser vendors are already pursuing an aggressive policy of opt-in feature enhancement, both through command line switches, config settings, and implicit and explicit flags (using HTTPS, meta tags, feature policy HTTP headers, etc.), I am not convinced by arguments that only in the case of ES versioning is it impossible for them to do so.

[bakkot]

@jonrimmer

they should provide authors with some mechanism, either opt-in or opt-out, by which code could choose which version of ES it wished to execute with

This is what we mean by "mode".

[domenic]

It's also worth noting that no browser implements specifications by versions. They implement individual features. The way these are packaged into yearly versions on ecma-international.org is an artifact of the process for securing intellectual property commitments, and has no relation to implementations. As such no implementer would be interested in encoding these IP-oriented distinctions into the actual language semantics. (To say nothing of the complexity of shipping multiple engine versions in the same browser, adding a new JS engine every year.)

[jonrimmer]

@bakkot Quoting from the ES2017 spec here:

The strict variant of ECMAScript is commonly referred to as the strict mode of the language. Strict mode selection and use of the strict mode syntax and semantics of ECMAScript is explicitly made at the level of individual ECMAScript source text units. Because strict mode is selected at the level of a syntactic source text unit, strict mode only imposes restrictions that have local effect within such a source text unit. Strict mode does not restrict or modify any aspect of the ECMAScript semantics that must operate consistently across multiple source text units. A complete ECMAScript program may be composed of both strict mode and non-strict mode ECMAScript source text units. In this case, strict mode only applies when actually executing code that is defined within a strict mode source text unit.

In order to conform to this specification, an ECMAScript implementation must implement both the full unrestricted ECMAScript language and the strict variant of the ECMAScript language as defined by this specification. In addition, an implementation must support the combination of unrestricted and strict mode source text units into a single composite program.

If what you mean by "mode" is, in fact, a pragma applied at a global level, operating across every source text file, and without the requirement to support the combination of different mode source text units into a single composite program, then it might be helpful to use a different term, given the one and only example of a "mode" currently used in ES is the complete opposite of these things in every respect. I would suggest the term "version switch", since that accurately describes what it does, whereas "mode" does not.

[bakkot]

a pragma applied at a global level, operating across every source text file, and without the requirement to support the combination of different mode source text units into a single composite program

That's not actually what you've described, though. A single execution environment can and does often execute interoperating code from several different pages (especially in the context of frames), which may have different meta tags or headers and therefore per your proposal different versions: this is the cross-realm concern I alluded to earlier.

In any case, sorry for the terminology confusion. The thing you're describing fits how the committee usually uses the word, but I can see why you'd think of it as a different kind of thing from strict mode and be confused.

[jonrimmer]

@domenic That's a fair objection. But I'm not demanding that browsers stop featurewise development, only that there is some mechanism for establishing a lineage of (potentially) backwards incompatible changes. A browser supporting opt-in to a particular compatibility level wouldn't imply that it supported every feature of that version, just a particular, minimal subset of backwards incompatible changes, and those incompatible changes shipped before it.

If a version number based on yearly spec releases aren't a good candidate for that, then there could be a separate versioning scheme for these kind of changes. The process could be that breaking features progress as normal proposals through the TC39 process, but browsers only implement them behind a command-line flag. At such point as there is one or more of these proposals that have been finished, they are assigned a compatibility tag, and browsers make them generally available using a pragma opting in to this tag. Using unique tags decided at a late stage, instead of a monotonically increasing number, would prevent developers declaring compatibility with unreleased/unfinished features.

So, the "flatten" proposal, might be assigned a tag "1.xkyt". Developers could opt-in to its changes via the pragma already mentioned. I.e

<meta name="es-compat" content="1.xkyt">

When a later proposal with backwards incompatible changes also release maturity, it would be assigned a new tag, e.g. "2.psra", "3.hlsu", and so on. Non-breaking changes could progress as normal through the existing process, and be implemented by vendors when they chose.

I know It's not the loveliest approach, but neither is contorting the API or being stuck with decades old bugs and design mistakes for compatibility reasons.

And I'm not sure why you're suggesting browsers would need to add a new JS engine for each change? As I write, Chrome has 277 feature flags in chrome://flags. Since I presume there are not 277 separate versions of V8 shipping in my browser, it seems like it is possible to support toggling features within the same engine implementation.