Support typescript with --experimental-strip-types

marco-ippolito commented 2 months ago

Add this issue to keep track of the work for supporting typescript out of the box, in the PR you can find documentation etc... The main idea is to use type-stripping. This is achieved ideally through the use of an external dependency, in my opinion @swc/wasm-typescript. I would like to have something basic and usable to propose to the public.

Tracking points to discuss:

[x] Support commonjs syntax
[ ] SourceMaps (enabling them with a flag)
[x] Type stripping on .cts
[ ] Add a flag to enable transformation to support TS only features (enums etc...)
[ ] Extension guessing (.js because of transpilation => but at runtime is a TS)
[ ] Transpiling .ts files in the node_modules
[ ] TypeScript linting
[ ] Externalize swc wasm

Roadmap: https://github.com/nodejs/loaders/issues/217

mcollina commented 2 months ago

I have two concerns (none of them blocking):

Is swc on board with following our LTS? This seems a massive dependency we should not take on lightly.
TypeScript does not follow semver, and the language evolves relatively quickly. Is this approach stable long term? Everybody seems always to be supporting the latest version of TypeScript.

My proposal in https://github.com/nodejs/node/issues/43818 was to have a strategy for doing this automatically but not vendor anything. Something along the lines of:

$ node script.ts
Typescript support is missing, install it with:
npm i --location=global typescript-node-core
$ npm i --location=global typescript-node-core
...
$ node script.ts
"Hello World"

An alternative (possibly better for long-term support):

$ node script.ts
Typescript support is missing, install it with:
npx ts-node-core

Learn about TypeScript support at: ...
$ npx ts-node-core
Detected Node.js v24.42.0
What typescript support would you like?
[ ] type stripping - swc@22
[ ] type checking - tsc@23

...Installing

$ node script.ts
"Hello World"

marco-ippolito commented 2 months ago

To answer to your points:

SWC seems to be a mature project, it is already used by Deno for our same purpose, has Apache-2 license which is fine. I'm using @swc/wasm, the wasm version to avoid compiling rust. We probably should ask the blessing from the maintainer @kdy1 and maybe release a version that we dont have to patch 😄
With typestripping we dont really care what version of typescript the user is using because we just remove it (unless there is some syntax not supported by SWC), we dont perform type checking at all. I think SWC moves quickly enough. Also asking the user to install a dependency is imho a suboptimal DX, users just want to be able to run node foo.ts

If this proposal is accepted we could also add hooks to customizate the transpilation, and many more things that come out of the box from swc

kdy1 commented 2 months ago

I can configure a publish pipeline for a separate package/binary, or a Wasm publish pipeline if node.js will use SWC for this task.

marco-ippolito commented 2 months ago

I can configure a publish pipeline for a separate package/binary, or a Wasm publish pipeline if node.js will use SWC for this task.

If we are able to ship the initial PR with v1.6.6 for the next updates that would be amazing, thank you

benjamingr commented 2 months ago

+1 for all the suggestions and also +1 for @kdy1 have nothing but nice things to say about him and swc + there is prior art in other runtimes.

GeoffreyBooth commented 2 months ago

If what we ship is type stripping, then we’re essentially shipping experimental support for the Type Annotations proposal. Which is good! I feel like that’s what does make sense to include in Node core, and I hope that proposal graduates and V8 adds support and then we can drop our support for doing it ourselves.

In that vein, maybe a flag of --experimental-strip-types might make more sense? To clarify to users that our intended scope is not to support all of TypeScript’s syntax or features, such as paths or enums or type checking; we just strip types and that’s it, and if they want more they can register module customization hooks from a userland library. And type stripping should be stable and unlikely to change as TypeScript evolves.

marco-ippolito commented 2 months ago

If what we ship is type stripping, then we’re essentially shipping experimental support for the Type Annotations proposal. Which is good! I feel like that’s what does make sense to include in Node core, and I hope that proposal graduates and V8 adds support and then we can drop our support for doing it ourselves.

In that vein, maybe a flag of --experimental-strip-types might make more sense? To clarify to users that our intended scope is not to support all of TypeScript’s syntax or features, such as paths or enums or type checking; we just strip types and that’s it, and if they want more they can register module customization hooks from a userland library. And type stripping should be stable and unlikely to change as TypeScript evolves.

Not against renaming the flag, but for this very limited features that we are adding, --experimental-typescript makes more sense because we are actually only supporting .ts files, (no tsx or other flavors), but once this lands and remove the flag, we will think on how to make it customizable, and increase the number of flavor, that would be a real type-stripping

GeoffreyBooth commented 2 months ago

This was discussed today in the loaders meeting:

I think there was general agreement that type stripping (but probably not more) is something that we would like to see in core, either soon or eventually.
@joyeecheung asked that @marco-ippolito’s branch be implemented as a package similar to Undici that can be developed outside of the core release cycle and eventually merged in when it’s mature.
To exist in core, support would need to both take advantage of being in core, such as being native; and avoid implementing features of TypeScript that may change faster than our release cycle (such as type checking, and any transforms such as enums, decorators, ESM-CJS transpilation and others).
@legendecas will schedule a meeting with @marco-ippolito and some champions of the Type Annotations proposal to discuss collaboration and next steps toward potentially implementing that proposal in Node core.

kdy1 commented 2 months ago

I created https://www.npmjs.com/package/@swc/wasm-typescript, which barely strips out the type.

And I wrote documentation for it at https://swc.rs/docs/references/wasm-typescript

GeoffreyBooth commented 2 months ago

which barely strips out the type.

How can this be used to just strip types and nothing more?

kdy1 commented 2 months ago

See https://swc.rs/docs/references/wasm-typescript

GeoffreyBooth commented 2 months ago

See swc.rs/docs/references/wasm-typescript

I saw that. What settings would I use to just strip types and nothing more? No support for enums, decorators, transpiling module import/export, etc. As in, if I wanted to use this to implement the Type Annotations proposal and error on any syntax that requires transforms.

kdy1 commented 2 months ago

It's not transform() of @swc/core. It's a completely different thing.

It

strips type
supports enum
does not support transpiling decorator (parsing/codegen is possible with a config)
import/exports are not touched, except type-only imports. (Including imports not used as values, per config)

These operations is one typescript set, but I can remove the enum part and/or handling code for type-only imports. I'll add options for them soon.

GeoffreyBooth commented 2 months ago

These operations is one typescript set, but I can remove the enum part and/or handling code for type-only imports. I’ll add options for them soon.

What I would like is for some option or collection of options where I can use SWC to implement the Type Annotations proposal, but nothing more. In particular, see https://github.com/tc39/proposal-type-annotations#intentional-omissions:

Intentional Omissions

We consider the following items explicitly excluded from the scope of this proposal.

Omitted: TypeScript-specific features that generate code

Some constructs in TypeScript are not supported by this proposal because they have runtime semantics, generating JavaScript code rather than simply being stripped out and ignored. These constructs are not in the scope of this proposal, but could be added by separate TC39 proposals.

Enums

Namespaces

Parameter properties

Omitted: JSX

JSX is an XML-like syntax extension to JavaScript that is designed to be transformed by a pre-processor into valid JavaScript. It is widely used in the React ecosystem, but it has been used for different libraries and frameworks. Because JSX directly interleaves with JavaScript code, compilers and type-checkers for JavaScript typically also support checking and transforming JSX as well. Some users may hope that the JSX transform could also be directly supported by ECMAScript, to expand the set of use-cases that can be handled without a build step.

We do not consider JSX to be in scope of this proposal because:

JSX is an orthogonal feature that is independent of optional static types. This proposal does not affect the viability of introducing JSX into ECMAScript via an independent proposal.

JSX syntax expands into meaningful JavaScript code when transformed. This proposal is only concerned with syntax erasure.

This section basically defines the parts of TypeScript that can’t be stripped: these are transforms, where SWC isn’t merely stripping these things out but rather injecting new JavaScript in their place. If our goal is just to implement type stripping, then we should throw exceptions when encountering any TypeScript syntax that requires transformation.

To be even more careful, we could also throw on the various items in https://github.com/tc39/proposal-type-annotations#up-for-debate, as these are questionable as to whether they can be safely stripped: stuff like the class private keyword. I think it’s better to be conservative and error on all of these if possible, and potentially add things back one-by-one if a particular syntax can be shown to be strippable safely.

The vast majority of TypeScript exists outside of these exceptions; most projects should be able to get just about all the benefits of TypeScript even without this handful of exclusions. The result of adding the ability to run Type Annotations Proposal-compliant TypeScript will be to encourage people to write TypeScript code that is compatible with a potential future where Type Annotations graduates to be part of the language itself. That in itself is a win for the entire ecosystem, as we begin the process of folding TypeScript into the spec and bringing the users along with it. And for Node in particular, once Type Annotations hopefully becomes part of the spec, V8 will implement it and then we can rely on that rather than maintaining this ourselves.

JakobJingleheimer commented 2 months ago

@GeoffreyBooth I get why you're suggesting erroring on things that result in code being generated. But as a TypeScript user, I have enums etc. and if this errors when running on my completely valid code, it's not fit for purpose. I would not change my code to support an incomplete feature—I would just use something else. I imagine that will be the case for many if not all effected users. And enums are not rarely used, so I think such a decision here is a square wheel.

aduh95 commented 2 months ago

@JakobJingleheimer I think the main target is new users / new projects, I don't think it's realistic (or useful) to try to support existing project – if you already have setup your tooling, why would you move to the experimental support in Node.js, when you can keep using the tooling that works? On the other hand, if you're starting, not having to setup tooling is huge, and if the tradeoff is to not use enums, that's really a no-brainer IMO.

marco-ippolito commented 2 months ago

This was discussed today in the loaders meeting:

I think there was general agreement that type stripping (but probably not more) is something that we would like to see in core, either soon or eventually.

@joyeecheung asked that @marco-ippolito’s branch be implemented as a package similar to Undici that can be developed outside of the core release cycle and eventually merged in when it’s mature.

To exist in core, support would need to both take advantage of being in core, such as being native; and avoid implementing features of TypeScript that may change faster than our release cycle (such as type checking, and any transforms such as enums, decorators, ESM-CJS transpilation and others).

@legendecas will schedule a meeting with @marco-ippolito and some champions of the Type Annotations proposal to discuss collaboration and next steps toward potentially implementing that proposal in Node core.

My view on this

I totally agree, lets just support type stripping
I don't see this implemented outside of core simply because there are plenty of alternatives, what we want to improve here is the DX. If this lands even as an incomplete feature, I'm very positive community will be supportive and will see a massive improvement. This imho just needs someone to do the heavy lifting of the first implementation.
I don't agree that this has to land as NATIVE but the goal is to move it to native side.
I will be happy to sync with Type Annotations proposal but I'm not gonna let this bogged down by a Stage 1 Proposal

I'm happy ok with renaming it --experimental-strip-types, IF we support real type stripping. Changed my mind, actually lets remove enums, maybe it will drive the community not to use them or be implemented in the language 😃

nicolo-ribaudo commented 2 months ago

Note that the type annotations proposal has different syntax from TypeScript, so I recommend not conflating the two even if your goal is to just strip TS types.

JakobJingleheimer commented 2 months ago

not having to setup tooling is huge

Setting up the tooling for this is trivial:

npm i nodejs-loaders

node --loader=nodejs-loaders/dev/tsx

Bam. Done.

marco-ippolito commented 2 months ago

not having to setup tooling is huge

Setting up the tooling for this is trivial:
npm i nodejs-loaders

node --loader=nodejs-loaders/dev/tsx
Bam. Done.

This is exactly what users dont want to do 😆 and we have to acknowledge that node foo.ts is what everybody wants

JakobJingleheimer commented 2 months ago

we have to acknowledge that node foo.ts is what everybody wants

Mm, I get that.

What about some middle-ground: I expect users regardless of project pre-existing or not will not accept (read: ridicule) the missing features we're discussing disallowing. SWC can already handle those (if I'm not mistaken).

Proposed compromise: an additional flag to enable them, like --experimental-generative-transpilation that enables enums, decorators, etc already supported.

marco-ippolito commented 2 months ago

currently my pr already supports all of that with the version of swc we are using. BUT it's not real type-stripping. --experimental-generative-transpilation is reasonable

GeoffreyBooth commented 2 months ago

Proposed compromise: an additional flag to enable them, like --experimental-generative-transpilation that enables enums, decorators, etc already supported.

We could potentially do this, or we could push such users into importing a library. I don't see us being able to ever unflag transforms because of the semver problem and TypeScript moving faster than we do and not being specified. For that reason transforms feel like a bad fit for core.

marco-ippolito commented 2 months ago

Proposed compromise: an additional flag to enable them, like --experimental-generative-transpilation that enables enums, decorators, etc already supported.

We could potentially do this, or we could push such users into importing a library. I don't see us being able to ever unflag transforms because of the semver problem and TypeScript moving faster than we do and not being specified. For that reason transforms feel like a bad fit for core.

We can give it a try and see how it goes

legendecas commented 2 months ago

It would be better to have TypeScript team buy-in to build in TypeScript syntax, particularly the syntax and transformation would be bound to Node.js release cycles. Ultimately, this type-stripping-only support in Node.js still requires TypeScript to perform type-checking to make it a complete DX. /cc @DanielRosenwasser @robpalme

Additionally, I didn't find how ESM/CJS support would work in the draft PR (e.g. it doesn't support transpiling CJS at the moment). I'd like to learn more about the support plan since TypeScript has Node.js targetted options like module: nodenext and there are open issues like https://github.com/nodejs/node/issues/50981.

JakobJingleheimer commented 2 months ago

I don't see us being able to ever unflag transforms

I was not envisioning ever unflagging them: either they make it to spec or they stay experimental.

Additionally, I didn't find how ESM/CJS support would work in the draft PR (e.g. it doesn't support transpiling CJS at the moment).

I think we should not support transpiling between CJS ↔ ESM at all whatsoever—it stays whatever it was authored.

marco-ippolito commented 2 months ago

Some of my thoughts:

If we want to use swc we would need:

cjs package without dynamic loading of the wasm
(Thinking out loud) maybe keep the source code in the repo and do the compilation to wasm during the dependency update because we want too see the source we are compiling and patch it if we need it. (this might be hard because swc is a big monorepo). If we keep the source code (but dont bake it), it might be easier to actually at some point add the rust to the build and bake it
Proper type stripping (no transformation)

What we would need from the typescript team:

Type stripping flag (this is currently afaik not implemented by the typescript compiler, isolatedModules only performs warning)
Figure out stability etc...

Why type-stripping and no transformation? I guess stability (?) as those are features without a spec (?) I guess its ok to avoid them. Probably offer a flag to instead perform the transformation

GeoffreyBooth commented 2 months ago

Why type-stripping and no transformation? I guess stability (?) as those are features without a spec (?) I guess its ok to avoid them. Probably offer a flag to instead perform the transformation

I think they’re almost two different features: type stripping is simply running TypeScript files (or a subset of TypeScript syntax) whereas “full” TypeScript is getting transforms, getting type checking, getting transpilation, getting paths, getting the ability to customize all the various options via tsconfig.json and so on. There are going to be people who want the full experience and they’re also not going to want to be locked on a particular version of TypeScript from years ago. I’m not sure there’s much value in something in between, like type stripping plus transforms but not type checking.

So we can add SWC and get type stripping now, and there’s a path toward stability for that; and we can separately work on improving the UX around adding the official typescript package as a local dependency for users who want the full experience in a way that’s better than today’s status quo but also doesn’t break our semver guarantees. (See https://github.com/nodejs/node/pull/49704.)

benjamingr commented 2 months ago

TypeScript is the most requested feature by our users, we have to acknowledge our own surveys. I support transpilation/type stripping through SWC and thing we should land and iterate.

I don't think we should limit ourselves to the type annotations proposal since that is still likely to change and would prevent adoption. I'm happy for Node.js to eventually decide whether or not to further limit the TypeScript syntax based on standardization efforts before the feature is stable.

JakobJingleheimer commented 2 months ago

I think getting TypeScript itself to do what we need is a no-go, not least of which because they do not follow semver. Also, it's super slow. I think SWC is way more viable.

I just noticed something in the TS docs, so I'm gonna play devil's advocate with myself:

https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums

In modern TypeScript, you may not need an enum when an object with as const could suffice

They are more or less recommending not using enums (and the alternative is just about the same). So if enums are getting phased out in favour of objects as const, actually maybe we're in the clear there and we can call enums "legacy" we don't support.

Decorators though 🤔 The proposal is stage 3, so perhaps native soon anyway (and thus not our problem).

Are there any other features that result in transformation? If none, I'm happy to get off my soapbox.

cspotcode commented 2 months ago

require('typescript').transpileModule() is the function for direct TS->JS transformation w/out typecheck, distinct from isolatedModules which is the typechecking companion but is a totally different thing.

acutmore commented 2 months ago

TypeScript type-stripping is something I have been actively working on at work (for internal purposes, not for landing in Node) so I'd like to just pass on some of the knowledge I gained from that.

It might be useful in both the implementation and in particular to make the validation easier. To be clear, this is just for passing on info that might be helpful to the folk doing the work here.

https://gist.github.com/acutmore/27444a9dbfa515a10b25f0d4707b5ea2

marco-ippolito commented 2 months ago

TypeScript type-stripping is something I have been actively working on at work (for internal purposes, not for landing in Node) so I'd like to just pass on some of the knowledge I gained from that.

It might be useful in both the implementation and in particular to make the validation easier. To be clear, this is just for passing on info that might be helpful to the folk doing the work here.

https://gist.github.com/acutmore/27444a9dbfa515a10b25f0d4707b5ea2

This is gold, thank you. I think white spacing is a very good alternative to source map that could work for us @kdy1 wdyt? Source mapping since its expensive behind a flag, white spacing by default so stack traces are in the right place

robpalme commented 2 months ago

Glad you like it. As the benchmarks show, ts-blank-space is very swift.

GeoffreyBooth commented 2 months ago

Simply replacing everything that’s stripped with whitespace, and therefore not needing source maps, is a brilliant idea. Is this something that could be added as an option to SWC? @kdy1

kdy1 commented 2 months ago

What a brilliant idea! @acutmore What did you use for codegen? You mentioned that you used typescript parser, but AFAIK it does not support preserving original line numbers.

Or, if not, did you do some lexer-level tricks?

@GeoffreyBooth I think it would be possible. I'm trying to implement retain_lines codegen option.

kdy1 commented 2 months ago

https://github.com/swc-project/swc/pull/9144 implements blank-space mode for codegen. Once it gets merged I'll publish a new version of @swc/typescript-wasm using it.

acutmore commented 2 months ago

What a brilliant idea! @acutmore What did you use for codegen? You mentioned that you used typescript parser, but AFAIK it does not support preserving original line numbers.

Thanks! All the generated code directly re-uses the original string. The codegen ends up being a loop alternating between

outStr += inStr.slice(endOfLastType, startOfNextType)
outStr += toBlankString(inStr.slice(startOfNextType, endOfNextType + 1))

magic-akari commented 2 months ago

@GeoffreyBooth @kdy1 ~~Should we report an error for unsupported syntax in SWC? If we leave the enum/namespace unchanged, the node parser will generate an error.~~

If we report an error in SWC, we can prevent a second parsing, thereby exiting the process sooner.

aduh95 commented 2 months ago

When just stripping types, ignoring syntax error is the safe bet, the next layer (V8) will either understand the syntax you don't know about (e.g. in case of a future syntax proposal), or throw the appropriate error.

GeoffreyBooth commented 1 month ago

Re https://github.com/nodejs/node/pull/53725#issuecomment-2211325041 by @DanielRosenwasser

I think the proposed behavior in this PR, where Node.js would claim to support TypeScript, but only a subset, would actually be a lot more confusing and lead to a lot of frustration in practice. . . . I do think it would be great to have a way to run TypeScript code in Node.js without a build step. But I think that the right way to do it is to not carve out a subset of TypeScript.

Hi @DanielRosenwasser, I’m replying to you over here as this is the “discussion” issue for the general approach, so that we can keep the comments on the PR focused on the implementation details of that PR.

I understand where you’re coming from: I fully expect that there will be users frustrated that the “strip types” feature doesn’t support enums, or extension guessing, or tsconfig paths, or module format transpilation, or any number of other features that TypeScript provides. That’s why we have https://nodejs.org/en/learn/getting-started/nodejs-with-typescript to guide people on using the module customization hooks to register userland libraries to handle “full” TypeScript, as user-provided libraries can operate on their own timeline with their own semver (or not) guarantees. We’re continuing to improve those hooks (see #201) and I hope that in the future we might have some level of built-in support for setting them up. For example, running a TypeScript file with an enum might (soon) error and provide the link to the guide page, but someday it might provide a command that the user can run to set up a TypeScript library that registers customization hooks to make that TypeScript file “just work.” That’s an approach that I can see passing muster with our project governance and our support time scales. I suggested in https://github.com/microsoft/TypeScript/issues/56838#issuecomment-1866999633 that the TypeScript team itself might want to provide official hooks that we could integrate with.

Put another way, Node 18.0.0 was released on 2022-04-19, when the current version of TypeScript was 4.6.2 according to https://github.com/microsoft/TypeScript/releases?expanded=true&page=2&q=2022. The Node 18.x release line is still active, so if 18.0.0 had bundled TypeScript then current 18.x releases would probably have that same version of TypeScript or maybe 4.6.4, assuming the latter had no breaking changes. TypeScript 4.7 had breaking changes, so we wouldn’t have been able to upgrade Node 18 to it.

I’m sure TypeScript 4.6.4 was a fantastic release, but surely there are users who would want to be running the latest and greatest TypeScript, so we’d be in the same position where we would need to provide something like the module customization hooks to allow a “bring your own TypeScript” approach, so that users could run the latest version. So really we’re faced with two options:

Bundle a version of TypeScript that will quickly grow outdated, and let users override it via the customization hooks.
Bundle type stripping which should never get outdated—and indeed, will hopefully be standardized and part of the language, and therefore eventually even supported in browsers!—and let users override it via the customization hooks.

Personally I think the second option is much more appealing both for us in the Node.js project but also for users. Type stripping is not only faster than tsc or any other transpiler that supports more features, but it’s also more compatible.

And hey, if we’re wrong and too many users complain, then we adjust. It’s an experimental feature and it’s flagged.

arcanis commented 1 month ago

That’s why we have https://nodejs.org/en/learn/getting-started/nodejs-with-typescript to guide people on using the module customization hooks to register userland libraries to handle “full” TypeScript, as user-provided libraries can operate on their own timeline with their own semver (or not) guarantees

It's kind of a failure if the feature is designed with the idea that people probably won't use it. This discussion is imo missing a lot of concrete user stories baking up your choices - the goal in https://github.com/nodejs/node/issues/43818 was initially "shipping something that would provide a better user experience for TypeScript users without additional configuration", not "shipping a polyfill for type stripping".

mcollina commented 1 month ago

@GeoffreyBooth you are missing the third option:

provide a "blessed hook" for typescript, so that running ts files always loaded it.

This is described in https://github.com/nodejs/node/issues/43818. Note that this approach is not conflicting with a "type stripping" solution: we can have a barebone implementation for scripts, and full customization capabilities.

marco-ippolito commented 1 month ago

@GeoffreyBooth you are missing the third option:

provide a "blessed hook" for typescript, so that running ts files always loaded it.

This is described in nodejs/node#43818. Note that this approach is not conflicting with a "type stripping" solution: we can have a barebone implementation for scripts, and full customization capabilities.

Or provide a flag that enables transformation of ts-only features since swc supports it

nicolo-ribaudo commented 1 month ago

While I wish that TS was only "JavaScript + types" and not "JavaScript + types + extra features", I think some of the backwards compatibility concerns are misplaced.

TypeScript often has breaking changes in their type-checking logic. Their transform logic is incredibly stable. There are two categories of TS-specific transforms (and things can graduate from 1 to 2):

TS-only features, such as namespaces and enums
Features that are similar to JS proposals/features but with different semantics

The output for (1) almost never changes:

The behavior of namespace has been exactly the same since TS 3.3 (the oldest version I can try in their playground), released in 2019.
The behavior of enum only changed once, in 5.0, but for code that would have been a type error until TS4.9 (so that, in practice, there was a big warning "do not do this" until then): https://www.typescriptlang.org/play/?isolatedModules=true&ts=5.0.4#code/MYewdgzgLgBAHjAvDARATRQbgFDYKZgCuAtjAIIwDe2MtMAGkvADQ11pMDM2AvrnnAAOIAE6xKPIA

The output for (2) does change, but through a process that is so painfully slow that, to avoid breaking changes, spec-compliant support for class fields in TS is still behind an option:

class fields will one day change the default from using [[Set]] (https://www.typescriptlang.org/play/?isolatedModules=true&ts=5.5.3#code/MYGwhgzhAECC0G8BQ1XQB7QLzQExIF8kg) to using [[Define]] (https://www.typescriptlang.org/play/?useDefineForClassFields=true&isolatedModules=true&ts=5.5.3#code/MYGwhgzhAECC0G8BQ1XQB7QLzQExIF8kg)
decorators, that people has been using through TS for years relying on a many-years-old version of the proposal. This was opt-in through an experimentalDecorators option, and the default behavior will be the spec-compliant one.

I believe that the discussion about "should Node.js transpile TS features other than types?" is meant to be centered about category (1), and as such you should only consider breaking changes that affect them while evaluating TS stability. Features can move from (1) to (2) (for example, if TC39 will start to actively consider an enum proposal), but it would take many years and Node.js would have multiple major release cycles to go through a full deprecation/removal of the soon-to-be-wrong feature.

(1)+types is what tools like Babel and SWC support. I'm not advocating that Node.js should do the same, but it's weird to see you conflating it with type checking (such as, in the "TS has breaking changes" proof linked in https://github.com/nodejs/loaders/issues/208#issuecomment-2212994179).

DanielRosenwasser commented 1 month ago

@nicolo-ribaudo, I really appreciate you providing that explanation and I just want to elaborate a bit more.

There's a lot of things one might refer to with respect to TypeScript - for example, the grammar, the type-checking semantics, the runtime behavior of its constructs, the loading behavior with respect to a given module strategy, the API of the compiler/language service, etc. But like @nicolo-ribaudo mentioned, a change in some of these may be entirely orthogonal to supporting TypeScript in Node.js.

For example, you referred to behavioral changes in an older version of TypeScript; but a developer who is "stuck" on TypeScript X.Y can still use a newer version of Babel, esbuild, swc, oxc, etc. These tools have been successful in implementing TypeScript and keeping pace in its releases.

For the above reasons, there's no reason Node.js could not support constructs beyond just TypeScript type annotations, even with an alternative compiler. On https://github.com/nodejs/node/pull/53725#issuecomment-2211325041, when I advocated that Node.js should not ship a stripped-down TypeScript, I was not advocating for shipping the TypeScript compiler and its API. You can truly support TypeScript by using any of the compilers mentioned above.

Finally, it feels like things are continuing to progress over at https://github.com/nodejs/node/pull/53725 with PR approvals even with the concerns we raised. Is there any way we can be involved in this effort and get into the specifics? I would really love for Node.js and TypeScript to get this right together.

GeoffreyBooth commented 1 month ago

Finally, it feels like things are continuing to progress over at nodejs/node#53725 with PR approvals even with the concerns we raised. Is there any way we can be involved in this effort and get into the specifics? I would really love for Node.js and TypeScript to get this right together.

We would be happy to collaborate on this. I’ve already added https://github.com/nodejs/node/pull/53725 to the loaders agenda, so it will be discussed at the next loaders meeting on 2024-07-16 at 11 am PT. Can you or anyone from your team attend? I put a block on the PR so it doesn’t land in the meantime.

Besides that, https://github.com/nodejs/loaders/pull/210 is a draft design document for the PR in progress. In particular I could use your insights in regard to https://github.com/nodejs/loaders/pull/210#discussion_r1669434926. We can also expand that document or create alternative documents to discuss approaches besides type stripping.

I don’t think anyone is advocating for type stripping to be the only way that Node supports TypeScript. There’s already the module customization hooks, that people can use with libraries like tsx or ts-node, and we have https://nodejs.org/en/learn/getting-started/nodejs-with-typescript to help people set those up. We absolutely want to make that easier and more automated, as discussed in https://github.com/nodejs/node/issues/43818 and https://github.com/nodejs/node/pull/49704, though we’re in the middle of redesigning the hooks per #198.

From my perspective, there’s an appealing simplicity to stripping types. It’s easy to explain and reason about, and should have fewer edge cases and bugs than a more elaborate solution; and there’s a path toward our implementation disappearing if the Type Annotations proposal graduates. If Type Annotations truly is the future, perhaps it’s not a bad thing to nudge people toward writing future spec-compliant JavaScript today; and the hooks allow using any syntaxes and features and options that users might want.

My questions for the TypeScript team are:

What features of TypeScript would you want Node to support in a built-in way that type stripping doesn’t cover?
Are the current transforms (enums, etc.) stable? Can TypeScript provide any kind of semver guarantee regarding them?
Is there any chance that TypeScript might add new transforms in the future, or new syntax that wouldn’t be strippable?
Does the TypeScript team discourage transforms in new code? Per https://github.com/nodejs/loaders/issues/208#issuecomment-2207358661, would it be possibly a good thing for us to exclude support for them as part of this new feature not needing to support legacy TypeScript?
Per https://github.com/microsoft/TypeScript/issues/56838#issuecomment-1866999633, would the TypeScript team possibly be interested in writing hooks for Node? This would essentially be an official version of tsx/ts-node.

DanielRosenwasser commented 1 month ago

Yes! How can we attend?

We haven't introduced new non-ECMAScript runtime constructs in years and steer away from them as much as possible. Nicolo gave examples of when they did change, but the impact was extremely limited/rare.

I'll try to answer your other questions after chatting with others on my team, but

Does the TypeScript team discourage transforms in new code?

can you elaborate on what you mean by this?

GeoffreyBooth commented 1 month ago

Yes, how can we attend?

It’s open to anyone. On Friday an issue like https://github.com/nodejs/loaders/issues/207 will get posted automatically. If you want to find me on https://openjs-foundation.slack.com I’m happy to chat there too. Our channel is #nodejs-loaders.

Edit: I just remembered I can't make next week, but I could do this Friday if that works for you. If you don't mind joining us in Slack, we can figure out a time without polluting this thread.

Does the TypeScript team discourage transforms in new code?

can you elaborate on what you mean by this?

I was referring to what @JakobJingleheimer wrote above in https://github.com/nodejs/loaders/issues/208#issuecomment-2207358661:

I just noticed something in the TS docs, so I'm gonna play devil's advocate with myself:

https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums

In modern TypeScript, you may not need an enum when an object with as const could suffice

They are more or less recommending not using enums (and the alternative is just about the same). So if enums are getting phased out in favour of objects as const, actually maybe we’re in the clear there and we can call enums “legacy” we don’t support.

I’m glad to hear that TypeScript is avoiding creating any new runtime transforms; can we go so far as to conclude that you discourage users from using the legacy transforms that already exist? As in, is there a future where the Type Annotations proposal graduates and TypeScript itself deprecates any syntax that can’t coexist with that proposal?

yyx990803 commented 1 month ago

Just want to echo @DanielRosenwasser 's concern from a user's perspective. When I read "It is possible to execute TypeScript code using --experimental-strip-types", I expect Node.js to be able to actually transform 100% of the TypeScript syntax, not a subset of it.

The Type Annotations proposal is not TypeScript. I would have no problem if the feature is documented as "Type Annotations proposal support" and the transforms being applied to .js files. However, if it's documented as "TypeScript support" and applied to .ts files, then it should work 100% like TypeScript.

And as @nicolo-ribaudo has pointed out, the concerns around TypeScript's versioning is conflating the breaking changes in type-checking behavior with that in pure syntax transforms. The syntax transforms are incredibly stable and multiple tools have been able to implement such transforms and have little trouble staying up to date. Given the breakage in syntax transforms are so rare and infrequent, and the fact that the TS team is steering away from introducing new runtime constructs and also willing to collaborate on making this happen, I don't think there are real downsides by going with full TS transform instead of just type stripping.

nodejs / loaders

Support typescript with --experimental-strip-types #208

Intentional Omissions

Omitted: TypeScript-specific features that generate code

Omitted: JSX