search

nodejs / node

Node.js JavaScript runtime ✨🐢🚀✨
https://nodejs.org
Other
105.1k stars 28.47k forks source link

How could we support typescript without vendoring it? #43818

Open mcollina opened 1 year ago

mcollina commented 1 year ago

I would like Node.js to support the following user experience

$ node script.ts
Typescript support is missing, install it with:
npm i --location=global typescript-node-core
$ npm i --location=global typescript-node-core
...
$ node script.ts
"Hello World"

(I picked the typescript-node-core name at random, I would be extremely happy if that could be ts-node)

Note that script.ts could evaluate to either cjs or mjs depending on the tsconfig.json file.

Originally posted by @mcollina in https://github.com/nodejs/node/issues/43408#issuecomment-1182883689

Why vendoring is not an option?

  1. TypeScript changes too fast and they do not follow semantic versioning, nor do they provide any long-term support.
  2. We will never be able to pick a default tsconfig given the 100+ options.
mcollina commented 1 year ago

cc @cspotcode

targos commented 1 year ago

So basically you would like Node.js to automatically load something from a global predefined place when it starts?

mcollina commented 1 year ago

So basically you would like Node.js to automatically load something from a global predefined place when it starts?

Yes, but only when started with a .ts file.

mscdex commented 1 year ago

Wasn't that the main purpose of custom loaders? You should even be able to set that in a NODE_OPTIONS.

GeoffreyBooth commented 1 year ago

Please don't start issues related to loaders without tagging @nodejs/loaders

mcollina commented 1 year ago

This is not related to loaders. This is about the developer experience of Node.js. Loaders are custom, user-specified components that are started before the app. I'm talking about shipping something that would provide a better user experience for TypeScript users without additional configuration.

Please keep this issue about the user experience and not specific implementations.

JakobJingleheimer commented 1 year ago

I'm trying to see how this isn't reinventing the wheel, but I'm not getting it. Could you please explain how it's substantially different?

How I'm seeing it is: This is already easily achieved via Loaders with next to no effort (and that effort is basically set-and-forget). We have a simple working example of it in node/loaders-tests/typescript-loader. For a more robust one, ts-node provides such a loader (ts-node/esm). And what we currently have via loaders is super fast. We just switched to it at my work and saw like an 800% speed improvement.

PS In case we veer into the territory: I would vehemently object to TypeScript support in Node itself.

mcollina commented 1 year ago

I'm trying to see how this isn't reinventing the wheel, but I'm not getting it. Could you please explain how it's substantially different?

I would rather stay away from discussing a specific implementation of this. This could be loaders but it doesn't matter. I care about listening to our users. Once we agree on the user experience, then we figure out what's the best way to ship it. If it's already possible as you hinted, it would be terrific.

PS In case we veer into the territory: I would vehemently object to TypeScript support in Node itself.

That's what our users are asking for. We cannot provide TS directly into Node.js core for plenty of reasons (it's just not possible for the tsconfig mayhem), including the fact that the TS team think it's a bad idea. I propose that we implement the user experience described in https://github.com/nodejs/node/issues/43818#issue-1303418894.

aduh95 commented 1 year ago

A few questions come to mind:

Currently, what happens when someone runs node script.ts depends on the env:

We would then need a way to detect that typescript-node-core program is installed on the local env, and to defer to it to load that file. Maybe by making a $PATH lookup?

GeoffreyBooth commented 1 year ago

In an app with "type": "module" in its package.json, you get this:

node script.ts
node:internal/errors:477
    ErrorCaptureStackTrace(err);
    ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /private/tmp/test/script.ts
    at new NodeError (node:internal/errors:388:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:80:11)
    at defaultGetFormat (node:internal/modules/esm/get_format:122:38)
    at defaultLoad (node:internal/modules/esm/load:21:20)
    at ESMLoader.load (node:internal/modules/esm/loader:431:26)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:350:22)
    at new ModuleJob (node:internal/modules/esm/module_job:66:26)
    at #createModuleJob (node:internal/modules/esm/loader:369:17)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:328:34)
    at async Promise.all (index 0) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

So to provide the UX you’re describing, we would have to add a special case within this error flow where if the unknown extension is .ts, we print a special message. This will inevitably raise the question of what other unknown extensions do we want to print guides for; or we could print some message for all unknown extensions along the lines of “go to https://nodejs.org/guide-to-loading-non-javascript-file-types” and that documentation page could be a clearinghouse of instructions for various types.

In an app without "type": "module", however, script.ts is parsed and executed as CommonJS. If it happens to be runnable JavaScript, it runs. Otherwise it’ll error when V8 tries to parse TypeScript syntax. So to provide a similar experience in CommonJS, within that error path you’d have to add a check that the file that couldn’t be parsed had an unknown extension, and then print the message. Keep in mind that there’s a type annotations proposal that would make many TypeScript files into runnable JavaScript.

Either way, the solution that we would be recommending to users would involve adding the TypeScript loader. And transpiling TypeScript is squarely one of the use cases for loaders. So I object to treating this as “not related to loaders” and removing that label.

aduh95 commented 1 year ago

@GeoffreyBooth let's imagine that the solution ends up looking like that (oversimplification):

if (entryPoint.endsWith('.ts')) {
  process.argv.unshift(entryPoint);
  entryPoint = '~/.node/typescript-node-core/bin.js';
}

Sure typescript-node-core will probably use loaders, but that's arguably an implementation detail. (FWIW I agree that a clean solution should involve loaders; if anyone is interested to work on this, that's where I would suggest them to explore)

JakobJingleheimer commented 1 year ago

What if it were some kind of…plugin (idk what the official term is relative to Node.js), like crypto or intl, with which node could be compiled or not? We could then have however many and we aren't responsible for them. It would be very similar to loaders, but a bit different and i think perhaps a little closer to what Matteo is talking about.

Also, this would avoid incorporating typescript-specifics (eg their .ts vs .js file extension nonsense) into node core (which is my main concern).

Annnnd it wouldn't require the CLI args everyone laments about loaders.

avin-kavish commented 1 year ago

Yeah, I think the important factor is user experience. Not sure how loaders work exactly but a DX like this would work - it could be specified in a configuration file which loader to use given an extension:

.m?ts --> use typescript loader
.cf --> use coffeescript loader
....  ---> any other user defined type

This could have global defaults and local (package.json) overrides. The global defaults would be shipped with node.js.

Maybe cli flags could be passed to the loaders, when needed. This way we can load any file conveniently with node path/to/file and spawned processes will also do so without additional config.

cspotcode commented 1 year ago

One thing to keep in mind: I see in comments this is described as triggered by the entrypoint's file extension. There are situations where you may be running a .js entrypoint but still want a TS-enabled runtime, for example my-cli which executes my-cli.config.ts.

...but that's a detail.

High-level ask from users is still: "How do I opt-in to my runtime having automatic TS support?"

cspotcode commented 1 year ago

When we say that users are asking for TS support, is there a sense for the ratio of desire for TS support vs desire for <insert non-TS thing here>?

benjamingr commented 1 year ago

I am in favor of this and I think we are not doing a good enough job. I like Matteo's proposed idea of a blessed "how to run TypeScript module" - ideally I'd also like us to provide a node build or similar on top of it so people have a workflow for CI.

We'd probably need:

Might also be interesting to know if this unlocks optimizations?

benjamingr commented 1 year ago

When we say that users are asking for TS support, is there a sense for the ratio of desire for TS support vs desire for ?

The sense in general as well as the trend from every other runtime is: Users are asking for better TypeScript experience a lot and almost no users asking for other type systems (like flow) or languages (like reason). Heck, someone opened an issue on this today

benjamingr commented 1 year ago

Also let me try pinging some TypeScript people

mcollina commented 1 year ago

When we say that users are asking for TS support, is there a sense for the ratio of desire for TS support vs desire for ?

Not at this point. I would be happy to evaluate other languages (like coffeescript), but I do not think any other language is as popular as TypeScript. Anyway, I think our implementation should allow for more extensions to be considered for inclusion.

bnoordhuis commented 1 year ago

More radical: bundle swc and just run the code. :-)

(Caveat emptor: swc is a transpiler, not a type checker. But on the flip side, it also handles JSX and TSX.)

JakobJingleheimer commented 1 year ago

More radical: bundle swc and just run the code. :-)

(Caveat emptor: swc is a transpiler, not a type checker. But on the flip side, it also handles JSX and TSX.)

That's not so crazy to me. It's sort of what I was suggesting above: https://github.com/nodejs/node/issues/43818#issuecomment-1183579118

manekinekko commented 1 year ago

After seeing how other emerging runtimes provide native TypeScript transpilation by default, I am in favor of @mcollina's proposed idea. I'd like to see Node support running TypeScript code, ideally out of the box, with no additional flags or options.

aduh95 commented 1 year ago

More radical: bundle swc and just run the code. :-)

(Caveat emptor: swc is a transpiler, not a type checker. But on the flip side, it also handles JSX and TSX.)

Does swc have a LTS policy compatible with Node.js one? If not, I don't think vendoring is an option.

benjamingr commented 1 year ago

@bnoordhuis

More radical: bundle swc and just run the code. :-)

Maybe sucrase? It's not getting as much traction but it's a pure JS tool and it's faster.

I also have positive experience with SWC

avin-kavish commented 1 year ago

When we say that users are asking for TS support, is there a sense for the ratio of desire for TS support vs desire for ?

yes, I did a community poll and 65% replied they use TypeScript in node.js. 35% picked JavaScript.

cspotcode commented 1 year ago

I appreciate that we're admitting it's by far requests for TypeScript and almost nothing else. Sometimes toys like coffeescript transpilers are used as justification that node's current APIs (loaders, etc) are sufficient. My experience is, they are not, and the challenges only arise when truly attempting to support typescript for a large number of projects. The focus on TS is healthy, because it will make node's APIs better. (and that includes being better for coffeescript and other stuff) EDIT the preceding paragraph is not great. I attempted to articulate my thoughts more clearly down here: https://github.com/nodejs/node/issues/43818#issuecomment-1184686030

In a few threads that ts-node's user experience is cited in favor of node's current APIs, --loader or otherwise. But we jump through a bunch of hoops to make that work, and node often makes it harder. Node's current APIs are not good enough.

Coming back to the idea of a blessed "how to run TypeScript module," I think we should make a list of all the hooks this module will require to do its job. One way to start: we can list all the hacks that ts-node goes through, all the places where we duplicate node's logic because node doesn't expose a proper API.

Here are a few off the top of my head:

The list is incomplete but hopefully illustrates the problem from a hook author's perspective.

avin-kavish commented 1 year ago

Maybe sucrase? It's not getting as much traction but it's a pure JS tool and it's faster. I also have positive experience with SWC

SWC and sucrase are both transpilers. To get the most out of TypeScript, they will have to be paired with a type checker.

It sounds to me like ts-node has already done most of the work. Couldn't it be combined with node to provide a seamless user experience?

cspotcode commented 1 year ago

Note that ts-node also supports the transpile-only swc/sucrase use-case. And avoiding typechecking does not avoid the need for the missing API surfaces described above.

Just want to clarify those things to avoid any potential confusion.

benjamingr commented 1 year ago

I appreciate that we're admitting it's by far requests for TypeScript and almost nothing else. Sometimes toys like coffeescript transpilers are used as justification that node's current APIs (loaders, etc) are sufficient.

Note, I personally value our users using other languages on top of the platform (be it flow, coffeescript, reason or anything else). I think it's important that a solution here would not hurt the development experience of those users. I also don't agree with the charactarization of CofeeScript as a "toy" (any more than Node is).

That said - absolutely: the feedback we've been getting from the community has overwhelmingly been in favor of TypeScript.

I also see a strong case for transpilation + swc over tsc and type checking by default since that's what the current proposal suggests.

cspotcode commented 1 year ago

Fair enough, and someone called me out on the same in a separate chat. I think I can clarify what I was thinking:

My belief is not that coffeescript itself is a toy, and "toy" was poor word choice to begin with.

I think there's a risk when discussing node hooking APIs to only look at simple use-cases and stop there. This can give the mistaken impression that an API is complete because it meets a simple use-case but not a more complex one. A coffeescript loader hook is an example of one that's simple to implement and does not have some of the complexity that exists in TypeScript hooks.

I think a focus on robust TS hooks will be healthy for node, healthy for fans of TS, coffeescript, flow, etc. Because it leads us to design feature-complete APIs that can handle demanding use-cases.

I'll put a little EDIT on my previous comment linking to this one, I hope this makes sense.

GeoffreyBooth commented 1 year ago

Here are a few off the top of my head:

@cspotcode This is a great list, thank you for compiling it. This is reinforcing in my mind that we should rename “loaders” to “hooks” and gradually expand the hooks that we offer to include other parts of Node that users might want to customize. Requiring users to monkey-patch is a hacky solution. Not all of these things would likely be solved with hooks, like some of the source maps ones could be improvements to the source maps API and we have the diagnostics channel for instrumentation stuff, but customization hooks could be a catchall general solution. This would also be a selling point for Node vs Deno and Bun, in that those other runtimes bake in their TypeScript etc. compilation which is certainly convenient but also means you’re limited by what the runtime offers; whereas Node will let users always use the latest (or any) version of TypeScript, or CoffeeScript or whatever else. I know TypeScript is all the rage right now but so was CoffeeScript eight years ago 😄

Copying over part of my comment from https://github.com/nodejs/node/issues/43408#issuecomment-1183626098, re the question here about how to make node script.ts possible. I think first we need to compare against the status quo. You can run this code today:

mkdir test && cd test
echo '{"type": "module"}' > package.json
npm install --save-dev typescript ts-node
echo "console.log('hello!' as string)" > script.ts
export NODE_OPTIONS="--no-warnings --loader ts-node/esm"
node script.ts  # Prints 'hello!'

So really the question is, what about this do we want to improve, and how? Providing the user with some kind of hint where they can figure out the above steps (or similar ones) so that node script.ts or node --loader ts-node/esm script.ts work for them? Or is the goal to eliminate some of these steps entirely, like by having Node automatically install some of these dependencies (and if so, which ones?). Or have configuration like a package.json define that a particular loader should be used, to avoid the need for --loader?

My answer would be to somehow define loaders via a configuration file or via part of package.json. We’ve discussed this off-and-on for awhile and it’s taken a back seat to just making loaders fully functional via the flag, but it’s an improvement I think we would welcome. The tricky part is enforcing that it applies only to the application scope and not its dependencies; like we want to allow the package.json that controls the entry point to define loaders (or any Node configuration?) but we don’t want to provide that power to any package under node_modules. This sounds like it should be easy to achieve technically but there are many gotchas (symlinks, etc.) and so we haven’t prioritized it. It also probably wouldn’t work for serverless platforms, unless we also provide a way to change Node configuration after the process has fully started up (and the serverless platforms allow that). So that also raises the question of whether we would want to ship an API that out-of-the-box doesn’t work for a major use case. Another idea would be an API to register loaders during runtime, like this:

import { registerLoader } from 'module'

await registerLoader('ts-node/esm') // Applies to any import evaluated *after* this registration
await import('./app.ts')

This would presumably work in serverless platforms but now this little entry point wrapper file is needed.

aduh95 commented 1 year ago

IMhO we want to remove the need for a flag at some point; and it would make perfect sense to me to have this kind of configuration in the package.json along side "type": "module". I'm imagining something like that:

mkdir test && cd test
echo '{"type": "module", "loaders":["ts-node/esm"]}' > package.json
yarn add --dev typescript ts-node
echo "console.log('hello!' as string)" > script.ts
node script.ts  # Prints 'hello!'

Much better than a global install if you ask me, because you could customize the TS version that is compatible with your project.

MoLow commented 1 year ago

it would make perfect sense to me to have this kind of configuration in the package.json along side "type": "module".

This sounds great but still does not solve some usecases such as REPL or node file.ts where there is no package.json

avin-kavish commented 1 year ago

I think at the moment typescript and types adoption is at a different scale compared to coffeescript 8 years ago, as evidenced by npm downloads and their types proposal that didn't get rejected. In my view, TS adds correctness guarantees to programming that no other type system adds all the while preserving the dynamic nature of the language; enabled by constructs such as unions, intersections, conditionals and type guards. At the very least, TS is the pioneer. I don't believe there will be a reversal in the trend of type safe programming. I find it reduces the write-test-debug cycles to a minimum, helps enterprise adoption (a la nestjs), makes refactoring easier, gives well defined APIs to packages and powers code completion in IDEs.

However, I do agree that node.js should allow for other styles of programming, so loaders can and will continue to enable this. I think the loader improvements proposed above are great and support those.

I think typescript users i.e. 65% of community (and growing) should be the priority. Every line of configuration is one extra overhead to manage. I think making the typescript experience at least as equivalent to javascript is justified, given the adoption rates. i.e. that's node just being able to run typescript. typescript package itself could follow node's module resolution rules, just as ts-node does. I think the implementation should have feature parity with ts-node, or users will just have to switch back to it. Which is why I suggested building on top of ts-node's work.

weswigham commented 1 year ago

A couple recommendations, or at least points to think about if you ever take this seriously and start designing an implementation:

Personally I think a higher value direction for the node maintainers, at least to start, is to be able to typecheck the node JS internals somehow (via external tool), just to build up some expertise with type-checked js code in the first place among maintainers, and maybe see some of the value the community gets from it. That way everyone working on node at least gets a good feel for what they'd actually want out of something like this, without throwing features at the wall and seeing what sticks (because "typescript support" has had differing meaning for every runtime I've read about). And if you then also start transpiling some internals as a followup step, maybe you'll find issues like https://github.com/nodejs/node/issues/41541 are actually critical to work through first, before even considering any kind of native support.

mcollina commented 1 year ago

@weswigham I was a bit thrown off by your message. I hope I'll clarify a bit what's the scope of this feature. Every single TypeScript user I've talked to is asking about some TypeScript integration with Node.js. The fundamental concept is providing some light-touch mechanism to serve the most basic case: node server.ts. This is useful for:

  1. development
  2. rapid turnaround of features

No one wants to vendor TypeScript with Node.js. I would like to provide the community with some hooks within Node.js to do what they are already doing with hacks & wrappers and provide a better developer experience.

tsc has over 100 compiler options to configure it. Some will be meaningful when transpiling or when importing. Supporting tsconfig files is probably critical. tsc's "zero config" checking and output isn't its recommended checking and output, for pretty much any situation nowadays. And you can't really change those defaults without breaking people and creating friction. We put "new defaults" into the output of tsc --init so new projects get better flags, so many projects have tsconfig files.

Unfortunately, I'm aware and this is one of the reasons we cannot ship TS in Node.js. No one wants to set a default tsconfig.json for all Node.js users.

tsc is pure overhead compared to plain js. You do not want to transpile or typecheck at runtime in a production grade application or library.

Agreed. In development, most people want to avoid friction.

Whatever process exists to execute TS should be capable of easily outputting equivalent shippable .js and .d.ts files like tsc itself (which is maybe more complex than you think when you start considering exports, imports, and output directories, and potential project references) when shipped via npm or the like - any .ts files shipped should only be for debugging experience (via source maps and declaration maps) and not for direct execution.

Why do you say so? Developers could just use tsc or other compilers.

Once you have a builtin way to optionally typecheck code (which, btw, should probably be invocable on js files, too, since tsc can typecheck most cjs and esm packages just dandy with a bit of configuration), you need a way to setup a watch mode for those changes, for instant feedback when developing. Some people will just use an IDE (which is also an important consideration - any configuration needs to be in a place where the IDE recognizes it, so ideally not a command-line flag to node), but some people like the command line watch experience. Same for transpiling into a shippable output. Maybe tsc itself works for this with a good implementation, but if deno taught me anything, that's not necessarily the case if you go off the rails with custom nonsense when writing your implementation, especially around module resolution. So take that as a warning, I suppose.

I do not understand this point. This issue was pretty clear that we are not planning in vendoring TypeScript and we will not provide anything built-in. Why developers can't just use the tools they already are? We don't need to invent anything new.

Typechecking is largely a whole-program operation. Meaning picking up files one by one an import at a time and checking them is error prone and often incorrect. TS has some flags to forbid some cross-file concepts; but the reality of programming is that there are global variables and shared state and files change them, and you need to reason about multiple files in concert to check them correctly. You should not be typechecking one-executed-import-at-a-time like the loader model initially suggests if you can help it - you should build a full program graph and typecheck the whole thing at once. (You can transpile one file at a time if you don't need declaration files, don't need experimental decorator metadata, and are willing to disable the const ness of const enums - all of those potentially need type information from the whole program - and const enums in particular are a meaningful performance improvement to a top-level application, even though we regret their type-directed behavior)

If my understanding is right, you are stating that it would be better to transpile the full application and then run the transpiled code instead of transpiling each file on its own via loaders. If I understood the various point of view, supporting "one file at a time" TS transpilation is a use case of loaders. @JakobJingleheimer, @GeoffreyBooth, and @nodejs/loaders could you chime in on this topic?

Personally I think a higher value direction for the node maintainers, at least to start, is to be able to typecheck the node JS internals somehow (via external tool), just to build up some expertise with type-checked js code in the first place among maintainers, and maybe see some of the value the community gets from it. That way everyone working on node at least gets a good feel for what they'd actually want out of something like this, without throwing features at the wall and seeing what sticks (because "typescript support" has had differing meaning for every runtime I've read about). And if you then also start transpiling some internals as a followup step, maybe you'll find issues like https://github.com/nodejs/node/issues/41541 are actually critical to work through first, before even considering any kind of native support.

This is not what this issue is about. Some progress has been made in this area and it's on everybody's radar. Are you stating that the Node.js maintainers 1. do not get the value of TS or 2. have never used TS? Maybe I'm reading this message wrong.

benjamingr commented 1 year ago

Hmm, maybe we should set up a call about it to brainstorm some strategies?

I'd like to invite @kdy1 to such a meeting (swc maintainer) and the ts-loader maintainers @johnnyreilly to discuss how an integration can look.

JakobJingleheimer commented 1 year ago

I quite like Antoine's suggestion (package.json) and think it could be implemented without much trouble.

No one wants to set a default tsconfig.json for all Node.js users.

Yes, we should definitely not do this. A user should be required to supply their own tsconfig options. I would be concerned though of users reporting typescript issues to us, for example tsconfig's paths (that could easily look like a Node problem when it's actually a fundamentally broken feature in typescript).

Why developers can't just use the tools they already are? We don't need to invent anything new.

Yes, I wholeheartedly agree it is better to allow users to choose which tool handles the actual transpilation. tsc is incredibly slow and there are several significantly faster tools.

If I understood the various point of view, supporting "one file at a time" TS transpilation is a use case of loaders. JakobJingleheimer, GeoffreyBooth, and nodejs/loaders could you chime in on this topic?

Yes, loaders handle one file at a time as the module graph is traversed. However, it doesn't necessarily have to transpile them one at a time. It's possible to collect all the relevant modules during the resolve() phase (probably a simple list/set) and then consume that whole collection in load() and merely supply the transpiled content for the current module one by one. I'm not sure if it's a good idea or understand the value in doing that, but you could. It would look something like

example loader that transpiles the whole app at once ```js const tsModules = new Set(); export async function resolve(specifier, context, nextResolve) { if (!isTypeScript(specifier)) { return nextResolve(specifier); } const { url } = await nextResolve(specifier); tsModules.add(url); return { format: 'typescript', shortCircuit: true, url, }; } let entireModuleGraphTranspilationOutput; // `load()` is not invoked until the entire `resolve` chain has finished // so all ts modules are guaranteed to have been collected already export async function load(url, context, nextLoad) { if (context.format !== 'typescript') { return nextLoad(url); } // This will make the `load` phase of the entrypoint very long, // but then the `load` phase of all subsequent modules will be instantaneous entireModuleGraphTranspilationOutput ??= await transpileEntireTSApp(tsModules); const { format, source } = entireModuleGraphTranspilationOutput.get(url); return { format, shortCircuit: true, source, }; } ```

Personally I think a higher value direction for the node maintainers, at least to start, is to be able to typecheck the node JS internals somehow (via external tool), just to build up some expertise with type-checked js code in the first place among maintainers, and maybe see some of the value the community gets from it.

Are you stating that the Node.js maintainers 1. do not get the value of TS or 2. have never used TS? Maybe I'm reading this message wrong.

That was my interpretation as well. I'm pretty sure every Node.js maintainer is familiar with both. I personally am eagerly awaiting the TC39 type annotations proposal.

Hmm, maybe we should set up a call about it to brainstorm some strategies?

I'd like to invite @kdy1 to such a meeting (swc maintainer) and the ts-loader maintainers @johnnyreilly to discuss how an integration can look.

ts-node's author, @cspotcode is an…adjunct member (?) of the nodejs/loaders team who has had good insight when we needed it.

kdy1 commented 1 year ago

swc does not have an LTS policy, but all rust crates of swc follow a very strict semver policy.

giltayar commented 1 year ago

My 2 cents about transpilation-only vs with type-checking: if we transpile only, without doing typechecking, we will gain three advantages:

  1. Much less reliance on tsconfig.json. Not sure we'd even need it for transpiling without typechecking.
  2. Less sensitivity to TS versions.
  3. Speed.

I believe 1 and 2 are very important and will generate much less noise, and 3 is a good thing to have, especially for production builds (where those needing even more speed will necessarily do the transpilation themselves during a build phase).

This is in alignment with the type annotations proposal, and IMHO the better option. Yes, the users will still need a "test:types" script that uses tsc for typechecking, but that's their choice, and they can choose whichever TS version and tsconfig.json they like, because it's the developer explicitly doing that and not an implicit version/configuration chosen by Node.js.

JakobJingleheimer commented 1 year ago

My 2 cents about transpilation-only vs with type-checking: if we transpile only, without doing typechecking, we will gain three advantages:

  1. Much less reliance on tsconfig.json. Not sure we'd even need it for transpiling without typechecking.

  2. Less sensitivity to TS versions.

  3. Speed.

I believe 1 and 2 are very important and will generate much less noise, and 3 is a good thing to have, especially for production builds (where those needing even more speed will necessarily do the transpilation themselves during a build phase).

This is in alignment with the type annotations proposal, and IMHO the better option. Yes, the users will still need a "test:types" script that uses tsc for typechecking, but that's their choice, and they can choose whichever TS version and tsconfig.json they like, because it's the developer explicitly doing that and not an implicit version/configuration chosen by Node.js.

That is what I currently do at work, and it works perfectly. The DX is such an improvement and the speed boost is 🤩

cspotcode commented 1 year ago

Tsconfig is still used to control certain emit characteristics, even when we can assume a certain level of ES support by the runtime.

On Fri, Jul 15, 2022, 6:01 AM Jacob Smith @.***> wrote:

My 2 cents about transpilation-only vs with type-checking: if we transpile only, without doing typechecking, we will gain three advantages:

1.

Much less reliance on tsconfig.json. Not sure we'd even need it for transpiling without typechecking. 2.

Less sensitivity to TS versions. 3.

Speed.

I believe 1 and 2 are very important and will generate much less noise, and 3 is a good thing to have, especially for production builds (where those needing even more speed will necessarily do the transpilation themselves during a build phase).

This is in alignment with the type annotations proposal https://github.com/tc39/proposal-type-annotations, and IMHO the better option. Yes, the users will still need a "test:types" script that uses tsc for typechecking, but that's their choice, and they can choose whichever TS version and tsconfig.json they like, because it's the developer explicitly doing that and not an implicit version/configuration chosen by Node.js.

That is what I currently do at work, and it works perfectly. The DX is such an improvement and the speed boost is 🤩

— Reply to this email directly, view it on GitHub https://github.com/nodejs/node/issues/43818#issuecomment-1185385709, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAC35OG5DZB74PW22DQRWMTVUEZFXANCNFSM53OVVUQQ . You are receiving this because you were mentioned.Message ID: @.***>

cspotcode commented 1 year ago

https://github.com/nodejs/node/issues/43818#issuecomment-1184765536

This sounds great but still does not solve some usecases such as REPL or node file.ts where there is no package.json

Agreed that this should be considered, and I would add the keywords shebang, $PATH, and shell scripting to consideration.

weswigham commented 1 year ago

Are you stating that the Node.js maintainers 1. do not get the value of TS or 2. have never used TS? Maybe I'm reading this message wrong.

Nah, nah, I'm moreso pointing out that it'd be way easier to know what a good design would look like for something like this if node core was in a better position to dogfood such a feature.

Much less reliance on tsconfig.json. Not sure we'd even need it for transpiling without typechecking.

You do. Lots of the options govern emit. Not as many as, eg, babel, but still plenty. It's definitely a place where tsc and tools like swc sometimes differ. Most non-tsc tools don't support transpiling const enums at all because they need type information, for example, which can be trading runtime speed for compile time speed; in our codebase, const enums improve our speed by a few percent in an average scenario.

Less sensitivity to TS versions.

It isn't none, however. For example, how a non-null assertion within an optional chain transformed just changed in 4.7 - a change few are likely to notice, but a change nonetheless.

Speed

That's mostly why I recommended some incremental-by-default build, and avoiding transforming libraries at runtime. The webpack plugin model, of forking and issuing type errors in a separate process from the one transforming, isn't the worst, either. Still, I imagine some people are going to want the option for a type error to block their build/run.

The more I think about it, the more I think something like a "prepare workspace" hook could do what's wanted here - something that can read and adjust the command line arguments before node actually interprets them and can do stuff like run a build in the workspace before starting the modified command. So it could change node script.ts into node out/script.js after building incrementally to disk. Gives the option to run the build fully through TS or whatever other tool, and optionally block the invocation if said build fails. Out-of-box, you could provide a passthrough hook that just scans the argument list for recommendations to install a specialized one into your workspace's package file.

GeoffreyBooth commented 1 year ago

There’s a meeting of the loaders team on Tuesday: https://github.com/nodejs/loaders/issues/96. Would any of you like to use that meeting to discuss this?

benjamingr commented 1 year ago

There’s a meeting of the loaders team on Tuesday: https://github.com/nodejs/loaders/issues/96. Would any of you like to use that meeting to discuss this?

I am not sure loaders is the right forum (is it?) since the ask here isn't to improve/change loader hooks but rather provide native integration. That said if you think that's on-topic then it might be the best avenue.

GeoffreyBooth commented 1 year ago

I am not sure loaders is the right forum

The loaders team is working on Node's support for TypeScript. Is any other team also doing so?

If not, I think the loaders team/meeting is the best place.

GeoffreyBooth commented 1 year ago

@benjamingr Can you join the Tuesday meeting? Do you want to invite the others you mentioned who you think would be good to include in a discussion?

I think that even if an eventual solution lives outside of the loaders API, the loaders team should be involved because the loaders API already provides a solution for this use case (see https://github.com/nodejs/node/issues/43818#issuecomment-1184715284). So the question becomes what about any proposed new solution makes it preferable to the baseline of what’s already possible today. And I think there certainly could be one; @aduh95’s suggestion in https://github.com/nodejs/node/issues/43818#issuecomment-1184727422 is very feasible, and we could also certainly do something around error messages that prompt users. I think the first question we need to solve is what DX we want, and then we can determine which parts of Node need updating to achieve that. The latter probably doesn’t need a meeting, at least for the solutions proposed so far, as they’re so simple.

mcollina commented 1 year ago

I won't be able to partecipate on that meeting. I encourage a "special meeting" for that, ideally under the next-10 umbrella cc @mhdawson.

mhdawson commented 1 year ago

Could everybody on the thread would would like to attend a "special meeting"

1) Indicate that they are interested and confirm if they can make 10-12 ET on July 28th or Aug 4th 2) Identify other people/teams we should mention/invite to the meeting.

In my mind the goal should be to discuss/agree enough so that we can update: https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-types-for-nodejs.md to add additional/goals agreed approaches.