Hijacking ignored type annotations for other features

[tolmasky]

I went into a little more detail about this in this twitter thread, but I'm curious about the thoughts here about using all this new "ignored" syntax for... other stuff. Given that this effectively adds a set of fancy, "structured", comments, that you can interlace throughout the code (and get nice syntax highlighting, etc.) for free, you could see it becoming a "syntax playground" of sorts. A set of syntax constructs that are effectively up the the user to decide what they mean. Obviously Typescript benefits the most from this since it can get away with the code functioning "identically" with these comments ignored, while other uses of these syntax constructs would still have to go through a transpilation step, but the real comparison point is how much easier it would be to prototype features when there is "free syntax" to use that (eventually) every JS parser would support. For example, you could whip up an implementation of decorators that just used the generic syntax. Instead of:

@decorator reactive
reactive method() { /*...*/ }

Which unfortunately requires you to make the difficult parser changes in whatever target transpiler you're using, you could instead just do:

method<reactive> { /* ... */ }

Anyways, I know I'd be tempted to do this (and hence have a weird desire to help push this through), but not sure what other thoughts are on this, especially considering that I think it kind of has to be allowed in order for the proposal to be truly "type system neutral".

[giltayar]

It's an interesting thought: you could use the "space" for types for whatever you want. But as you mentioned, any substantial syntax would need to be transpiled. The whole purpose of this proposal is to remove the need for transpiling TS/flow/etc.

So, in theory you are right. But what you propose could be done today without needing this proposal, but just writing a suitable transpiler.

[orta]

Yeah, one of the downsides using these spaces for interesting syntax is that it's not possible to put evaluated code in there - so it's only useful to tooling outside of the actual runtime evaluation. Could make for interesting docs, but as Gil mentioned - it'd require a transpiler to insert the useful eval-time code

[tolmasky]

I answered on Twitter too, but for anyone reading this thread:

The idea here is that this is actually meaningfully different than what you can do today, and dramatically lowers the bar on creating new feature transformations. You actually cannot trivially add new features that don't collide with existing JavaScript today. You for example can't just ship a new syntax plugin for Babel on npm, since Babel does not expose the API to do this publicly (all the parser internals are private). As far as I'm aware, for all the popular parsers, to add new syntax you have to fork the parser. They don't expose the actual parsing internals, only libraries to transform already-parsed code (and thus, libraries to transform existing well-defined AST nodes to other existing well-defined AST nodes). Beyond being difficult to add a new feature that isn't a semantic change on top of existing syntax, the fact that you need to fork the parser makes it a non-starter to share the code. It's way harder to get people to npm install babel-my-fork (that I have to keep up to date with Babel or else it will quickly become outdated) than it is to get people to npm install babel-plugin.

The key difference that this would open up is the fact that we are adding new AST nodes, that are guaranteed to not be used by JavaScript proper. It's like taking the old idea of "future reserved words you're not allowed to use", only, "future reserved words we WANT you to use!". It's as if we've added "hooks" to the language for you to "plug in" whatever functionality you want in a way that is guaranteed to be future compatible with any new JS syntax we ever introduce. The comparison point is not "but that is way harder than what Typescript is trying to do," it's "oh, that's way easier than what people currently have to do, both from an implementation perspective and a distribution perspective".

Sure, you have to write the transformer... but that's the easy part! People already write transformers for all sorts of stuff (hoisting inner functions that don't form closures for performance, minifying, whatever). It's easy (and encouraged) to go AST -> AST. But what you don't see a lot of is actual new language features, precisely because its complicated to understand how to add non-conflicting new syntax, and again, it's practically impossible to add it in a way others can use it because the public parsers don't have a "syntax" plug-in mechanism. This changes that. Day 1, you have Flow and Typescript "sitting" on this syntax, but tomorrow I can release my library that lets you do function f<memoized> { } or whatever, and at that point, if your community is JS-only and rarely interacts with Typescript, there's no perceived conflict there, and it looks way better than function f /*memoized*/ () { }, if for no other reason than editors would already have nice syntax-highlighting facilities for this sort of thing. In other words, we're creating "free real-estate" in the language, that on the one hand tries to not be too opinionated (I think everyone would agree that we wouldn't want to give typescript a leg up on Flow with this, or vice versa), but on the other hand relies on a "handshake agreement" to not be used for other stuff, despite being incredibly expressive.

To be clear, I find this as potentially exciting. I just want to throw out there that we shouldn't be surprised if people don't shy away from using this in weird ways that then if they take off end up eventually bringing different libraries into conflict with each other. I think if you look at the number of transformation plugins, compared to the number of syntactic additions, it points to the possibility that the main bottleneck is not that people don't want to add features, but that it's currently artificially difficult to do so compared to just changing the meaning of existing code through transformers.

[shaedrich]

Correct me if I'm wrong but isn't this proposal currently only stripping out specific TS syntax? Everyone now gets excited to strip out their own syntax. But for this to work and to know which of their syntaxes can actually be stripped out without problems, we would need to know a generalized way to do this, be it with hooks or the like.

If we allowed this to be used by everyone for everything, we need to be aware, that this will not just hypothetically lead to collisions, it most certainly will. Documentation then needs to be clear on this ("use at your own risk and deal with the consequences, you have been warned").

[ljharb]

The proposal isn't stripping anything, it's explicitly codifying it as a noop.

[shaedrich]

But I assume, JS still want to throw syntax errors and such in the future. So it has to—as you say—explicitly know which are syntax errors and which are none of its business

[ljharb]

Correct.

[shaedrich]

And if I'm not mistaken, there's some idea of how that could look for TS, yet no generalized one for other implementations, right?