Open mcollina opened 2 years ago
I'm interested and can attend that time on either day.
10-12 ET
~Morning on the US east coast is night time in Europe; if there's a time slot compatible with Europe time zone~ EDIT: I mixed up the timezones, nevermind. I'd be interested :)
@aduh95 10-12 ET is 3-5 pm in France I think. It's alreay starting at 7 AM for anybody in pacific time (west coast NA) so seem like a reasonable balance between West coast North America and Europe ?
That would work for me
Am I invited too?
Not sure because of my English skill
I'll try to attend, if I can. 🙂
Interested. Both times would work for me.
Would it be possible to attend this meeting as a silent spectator? I'm not affiliated with Node.js or TypeScript, but deeply curious how the process works and what gets decided on the topic.
@meyfa the meeting will be open and you are welcome to come and listen/participate.
@kdy1 invitation is open.
Can attend - multiple other Vercel folks may attend as well. Once a meeting invite is created I'll be sure to share internally
Created issue for a mini-summit here https://github.com/nodejs/next-10/issues/149
Hi, I'm the author of Sucrase. I won't be able to make the meeting, but am definitely interested in DX improvements here and wanted to leave a few thoughts:
load
hook, but even more focused on this use case).isolatedModules
flag to allow file-by-file transpilation, and all transpilers except tsc need that flag enabled (and tsc is much faster with it enabled). The TC39 type annotations proposal is also taking the "useful subset" approach. I took a look through all the options, and I think that if you assume that the goal is to execute TS (rather than output .js files) and assume isolatedModules
, no syntax downleveling, and no ESM -> CJS conversion, the only significant open questions are what to do about JSX config and import elision, which to me seems solvable. I realize I'm going against the TS team here, just want to share my perspective. 😄 (I still agree with the other reasons to avoid vendoring TypeScript.)@alangpierce that's great to hear about the Sucrase plugin. Can we sync up about your plans, either by talking in a sucrase issue or a ts-node issue? I had been worried about biting off more than I can chew with implementing additional transpiler plugins myself, but Sucrase was on my list because it fits a need for a smaller pure-JS option. So if there's anything you need from my end to make it a success, I want to help.
I mentioned in the call some projects which have the ability to use TS as config files, but where that experience is difficult, and @GeoffreyBooth asked me to post some of that info here.
Webpack does (and it's a lot of instructions). That uses interpret and rechoir, which let you require
files of other types.
vite supports vite.config.ts
by directly calling esbuild.
I couldn't remember the name of the library I was thinking of that lets people use .config.ts
files more generically (it wasn't rechoir), but you can see more examples by doing a code search for config files with TS extensions.
EDIT: I was thinking of rc-config-loader
and cosmiconfig
.
This is a good example of where node can offer better first-class support. Interpret, rechoir, etc use CommonJS require.extensions
hooks, so they don't work for ESM. More generally, they assume the ability to install hooks in-process.
This has fragmented how tooling (the consumers of rechoir) have decided to support ESM.
For example, I believe some that call out to esbuild are actually bundling the config file with its deps, writing it to a temp location on disk, and then executing that script in-process.
Others decide to spawn a child process with --loader
for the config file, then marshal the config object across process boundaries as JSON. Webpack config can't do that since the config file has executable new PluginClass()
stuff in it.
This potentially relates to how we support hook configuration in package.json.
node ./foo.ts
can pull foo's package.json
But mocha ./foo.spec.ts
is calling an entrypoint that's outside of the project, which is in turn trying to execute code inside the project.
I was distracted by work at the tail-end of the call, so perhaps these concerns were already noted.
I couldn't remember the name of the library I was thinking of that lets people use .config.ts files more generically
Maybe liftoff?
Just stumbled across this thread and wanted to share TypeScript Execute (tsx), which I think offers a very similar developer experience described by OP:
$ npx tsx script.ts
"Hello World"
# If globally installed
$ tsx script.ts
"Hello World"
It's zero-config (no tsconfig.json
necessary) and automatically accommodates CommonJS/Module package contexts.
It's powered by esbuild and uses @esbuild-kit/esm-loader
(Node.js loader) & @esbuild-kit/cjs-loader
(require()
hook) under the hood.
I would love to see further TypeScript integration in Node.js, and I think tsx demonstrates the demand and practicality for tools like it.
Mm, I stumbled upon tsx
recently. Not to digress: I couldn't figure out how it's actually different from ts-node
(it seems the only difference is its claim to be faster than ts-node
because it uses esbuild
(go) whereas ts-node
uses tsc
which is JavaScript, but that's not really true—ts-node
has an swc
(rust) compiler option).
Advocates seeking more support with TypeScript have explicitly stated that somePackage script.ts
is too much trouble for end-users, so the example of tsx script.ts
unfortunately doesn't fit the bill (something like that was an early suggestion).
The no-config part might appeal though (I think esbuild
itself does not require a tsconfig?).
I elaborated on the comparison in the FAQ. Hope it clarifies the differences—happy to answer questions if you open a discussion in the repo.
I shared tsx for those that aren't convinced that Node.js's DX can improve dramatically by supporting a zero-config TypeScript runtime. Perhaps they can be persuaded by trying it.
where does this stand? is there anyone working on this / anyplace where this is tracked?
where does this stand?
Per https://github.com/nodejs/next-10/issues/149, the document https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-types-for-nodejs.md was updated with the consensus next steps: https://github.com/nodejs/node/commit/dcc9589e5ed36d511466fd43ea1f4b9c1f567899.
is there anyone working on this / anyplace where this is tracked?
Not that I’m aware of, but you can comment here (or open a new issue) if you’d like to start implementing some of the steps listed in https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-types-for-nodejs.md#high-level-approach—development-workflow and we can coordinate effort.
I can update the format error message; the doc it cites needs to exist first though.
If the file was a TypeScript file, a TypeScript specific message with a reference to a link on Nodejs.org specific on learning how to configure TypeScript will be provided.
I'm think a subsequent enhancement to this could be to "register" file extensions with their specific doc entry if it exists, and use that decide whether to show a generic or specific link
Ex
if (isUnsupportedFormat(url)) {
const link = thirdPartyFormats.get(ext) || genericLink;
const message = `…`;
}
We could give it its own error code and add it to the existing error code list: https://nodejs.org/api/errors.html#nodejs-error-codes
That would give us a path to eventually make unique pages per error code with more detailed and up-to-date information.
Hey folks, I was in openjs summit's node.js tooling group session yesterday where this came up and tried to come up with a new idea for how this could work.
To set up my priors, there's a few reasonable tensions at play:
useDefineForClassFields
(as mentioned by wes above)
I think a potential answer for out of the box index.ts
support in node.js is that the "typescript"
npm module adds support for the ESM loader APIs, and node.js adds support which looks to see if "typescript" is installed locally via node_modules (or globally) and automatically run the esm loaders for *.ts
, *.tsx
, *.mts
and *.cts
.
It puts some onus on both projects to move a little bit, but neither need to be fully committed to a vendored-ish connection where they are blocking each others work.
If the "typescript" lookup fails, you can recommend someone install "typescript" via npm. Given the need/desire to type-check *.ts
files, it's pretty likely this is already set up.
The pile of caveats:
node index.ts
because it requires the files to be ESM meaning there must be a package.json - is that an acceptable trade-off? Note, that I'm not saying that adding ESM loaders support is trivial for the TS team - but it is reasonably sized looking at the size of the support which was added to ts-node (~400 LOC, which then reaches into a bunch of pre-existing code which was already necessary for prior similar work)
I'm in favor of @orta proposal. I have some questions:
Does this even work for node index.ts because it requires the files to be ESM meaning there must be a package.json - is that an acceptable trade-off?
Why? I don't see why this would be needed.
Performance oriented JS transpilers (sucrase/swc/esbuild/etc) would likely be faster than the "typescript" transpile APIs, but at the trade-off of perfect accuracy - this doesn't proclude them being used as loaders, but does prefer the built-in TypeScript. It feels like a push to say that node could have a list of their loaders are prefer them, but I think there'd be resistance in node.js to that given the dancing landscape of the JS ecosystem.
We should provide some configurability for this mechanism, something in package.json
that allows us to map .ts
to the swc
loader.
I assumed that because they were called ESM module loaders they would be ESM only - but if not, that makes a stronger case for this technique.
The configurability makes sense, as "typescript" would be default any other loader-ish package definition should be able to overwrite it 👍🏻
I assumed that because they were called ESM module loaders they would be ESM only - but if not, that makes a stronger case for this technique.
I think there is a plan / possibility to extend them to CJS too.
Hey folks, I was in openjs summit’s node.js tooling group session yesterday where this came up and tried to come up with a new idea for how this could work.
Hi @orta, thanks for your ideas. A few thoughts:
Per above, we have a few next steps already outlined: https://github.com/nodejs/node/blob/main/doc/contributing/maintaining-types-for-nodejs.md#high-level-approach—development-workflow. These are pretty minimal though (an error message that directs people to instructions) so there’s no reason we couldn’t start with these improvements and continue onto more dramatic enhancements later.
Your idea sounds a lot like https://github.com/nodejs/node/pull/43973, a PR for adding loaders configuration to package.json
; note the last few comments on that PR. Those ideas (support .env
file, or a Node configuration JSON file) could be ways to achieve what you’re after, without needing to add anything particular to TypeScript into the Node codebase. Something like an install script could automate adding the configuration.
I wouldn’t want Node to treat a "typescript"
dependency as a special case where its presence makes Node behave differently. Besides being potentially confusing (it seems to break the rules of what Node modules can do, if this one is treated differently than others) it also means that Node only has this enhanced TypeScript support when the primary "typescript"
package is installed; but users might very well prefer to use esbuild
or tsx
or who knows what else for their TypeScript compilation and/or type-checking. However as long as we have npm install scripts, npm install typescript
could easily prompt the user something like “add configuration to support TypeScript?” and then it could do something like add --loader typescript/register
somewhere (package.json
"scripts"
, a future .env
or config JSON file). This would be a general solution that could work for anything that would want to customize Node via a loader (JSX, YAML, CoffeeScript, whatever). I know there are security concerns around npm install scripts, but npx
commands could do the same work; there could be a TypeScript equivalent to npx create-react-app
that would add the dependency and configure the options.
Loaders/Hooks/Customizations are still being figured out but having this as a launch goal could be a big ecosystem win, and could help shape their API?
Loaders are on the verge of being stable: https://github.com/nodejs/loaders#milestone-1-parity-with-commonjs. Adding support for some type of file-based configuration (as opposed to the current flag-based) is on the roadmap, but unnecessary for the Loaders API to leave experimental status because it shouldn’t be a breaking change to add a new way of defining configuration. There are no API changes needed to support TypeScript, as it works today via things like node --loader ts-node/esm
. All that remains is making this easier, either through prompting the user with what to do or automating some or all of the steps to making it work.
Also, a solution along these lines, where we don’t special-case the typescript
package, means that we don’t need any buy-in from the TypeScript team. We could solve this all on our own, and users could choose whichever TypeScript library they prefer.
There are no API changes needed to support TypeScript, as it works today via things like
node --loader ts-node/esm
.
This is a bit misleading since ts-node/esm
installs several non-loader hooks into the main thread. Yes, it does work today, but future changes to node or to the loader API might break or hinder our ability to install these non-loader hooks. From a user's perspective, it's a loss of functionality when those hooks go away: TS that was working previously will start failing with runtime errors.
Yes, it does work today, but future changes to node or to the loader API might break or hinder our ability to install these non-loader hooks.
I’m unsure what this means. We’re not intending to change Node to break TypeScript support. On the contrary, making the Loaders API stable is meant to achieve the opposite: an API that can be relied upon, that won’t be removed or changed absent a Node semver-major change.
We have discussed adding similar customization APIs for things like source maps and the REPL and so on; I assume these are the “non-loader hooks” you refer to? The goal there would be to add APIs to allow tools like ts-node
to customize those areas of Node in non-hacky ways that also provide more stability, as they would be officially supported Node APIs rather than unofficial monkey-patching or the like. But again, no one benefits from changes to Node causing breaks in expected behavior. The goal would always be to improve compatibility and reliability.
I’m unsure what this means.
We have to use undocumented monkey-patches. When node ships breaking changes to undocumented monkey-patchable functions -- perhaps accidentally -- there is sometimes resistance to fixing the breakage, since the API surface is undocumented. Stabilizing loaders is good, but it doesn't stabilize these undocumented hooks. From a maintainer's perspective, this risk is not fun.
One of those hooks is to give us proper module resolution: Module._resolveFilename
It's not related to REPL nor sourcemaps.
I agree everyone's on-board to enable great TS support. I hope that until the loaders API fully supports the necessary hooks, the node team will treat these undocumented monkey-patches as an API surface that cannot be broken.
I hope that until the loaders API fully supports the necessary hooks, the node team will treat these undocumented monkey-patches as an API surface that cannot be broken.
The undocumented hooks you’re talking about are additional scope beyond the Loaders API, but something we intend to address with a new Customization API: https://github.com/nodejs/loaders/issues/95. The goal is to support things like ts-node
entirely through documented, stable APIs.
These days it is common to use services that do provisioning and management for you, AWS Lambda in particular. So they do not care that much about node.js runtime. But developers do care. Developers are forced to transpile the code before deployment and have transpiling pipeline.
Secondly, I can't believe external transpilation could do better than native execution.
Thirdly, bun and deno already have this feature. Their runtimes want to stand out and get some audience based on unsolved needs. Isn't it a good sign that market expects TypeScript to be there?
I think we can have pre-defined values of tsconfig and the developer who wants to use other resources create inside the folder manually.
Ok, I started to experiment with implementation and here are my findings/opinions for tonight:
% ../node index.ts
The TypeScript compiler is not available.
Install it following documentation at https://nodejs.org/en/knowledge/getting-started/working-with-typescript/
then
% ../node index.ts
1
number
if the @node/tsc-adapter
module is available in the path - it can be local or global
.ts
file from node_modules
or at least allow it behind a flag, this can't go well because of potential configurations missmatches% docker run -it --init denoland/deno:1.10.3 repl
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Deno 1.10.3
exit using ctrl+d or close()
> const x: number = 1
Uncaught SyntaxError: Missing initializer in const declaration
.ts
extension. I guess that since for 99.9% of the cases, semantically, TypeScriptTranspile(jscode) === jscode
we could just ignore this problem?I won't have coding time for a couple days, but should go back into writing stuff early next week
@vdeturckheim This looks good! A few thoughts:
@node/tsc-adapter
will contain, but I assume it’s a TypeScript loader like ts-node
? If so it’s tied to the Loaders API which is still experimental, but #44710 is the last PR before it could become stable. Also this TypeScript support will surely be experimental for a while too, and longer than the Loaders API is, so that shouldn’t be an issue.@node/tsc-adapter
package would be installed and versioned separately from the Node runtime, it would have to support a range of Node versions. I think this is a problem already solved by ts-node
and similar projects.package.json
stuff, there was a prior discussion about including the loader settings (like what would be passed into --loader
) in there, and I remember the consensus landing on including all of NODE_OPTIONS
in there, or as much as is feasible. I think the way forward is to create something like a package.json
"nodeOptions"
field and add as many options as possible; it’s probably impractical to support all 100+ in a single PR, but we could at least come up with a design that can accommodate eventually supporting all of Node’s options and we can add them in batches..ts
extensions isn’t so weird, they’re already required in ESM JavaScript generally. I know it’s unidiomatic for TypeScript, but support for explicit .ts
extensions is coming in TypeScript 5.0 and we can provide an informative error on missing .ts
extension like we already do for missing .js
extension.Thanks @GeoffreyBooth , these are very good points!
Yeah, @node/tsc-adapter
is another name for typescript-node-core
suggested in the first message of the thread and would be ts-node with node using it instead of having to plug into node. Basically, making the TS compiler live outside of node while allowing node xx.ts
without any other argument. Right now, I made it work with cjs but for esm it will likely be a loader (we had experiments with @targos a few years ago on that too).
Regarding node vs. package version, this is a very good point. I think we are going to a direction where the typescript support can work very well in most cases and where we provide DIY solutions for when it does not (like overriding the typescript adapter for instance).
For repl and .ts
cases. I agree 💯 with you. In the end, these are best-effort and DX choices, we can take the decisions and document them. Also, this is what experimental support is for!
I missed last summer's summit (and this thread until recently fwiw), so I hope I am still going in the direction people agreed upon 👼
I missed last summer's summit (and this thread until recently fwiw), so I hope I am still going in the direction people agreed upon 👼
Yes, I think nothing has changed from https://github.com/nodejs/node/issues/43818#issuecomment-1264954915 and what your recent experiments seem to assume. About the only “news” is that we’ve made so much progress with the Loaders API that I can see it applying to CommonJS too in the not-too-distant future; so I would prioritize getting TypeScript support via Loaders working first or at least ship that at the same time as CommonJS solution. @cspotcode and others have also had discussions about hooks in addition to those provided in the Loaders API, to do things like customize the REPL, for a more seamless DX beyond loading modules. Hopefully we can use the interest in better TypeScript support to build out some of those APIs.
It sounds like package.json
options aren’t needed for the core/MVP use case, so maybe we should get the initial to-do list done and then we can loop back to that and give it the attention it deserves.
This sounds good, thanks for the highlights! Seems good to me! I'll resume work with that in mind!
Regarding the point in not treating "typescript"
differently (in https://github.com/nodejs/node/issues/43818#issuecomment-1264954915), I agree, that's one of the advantage of having something like @node/tsc-adapter
which we control and would only serve the goal of calling "typescript"
from node under the hood on .ts
files. But nothing prevents to also have @node/swc-adapter
or @node/esbuild-adapter
tooto let the end user chose the transpiler in a list of provided alternatives (these packages are proxies to the other packages which also abstract breaking changes such packages could have). The end user will anyway be able to use a custom loader if they want to use something else in the end.
But let's focus on the MVP for now and add too much complexity later 👼
But nothing prevents to also have @node/swc-adapter or @node/esbuild-adapter tooto let the end user chose the transpiler in a list of provided alternatives (these packages are proxies to the other packages which also abstract breaking changes such packages could have). The end user will anyway be able to use a custom loader if they want to use something else in the end.
+1 on this ^^
But nothing prevents to also have
@node/swc-adapter
or@node/esbuild-adapter
tooto let the end user chose the transpiler in a list of provided alternatives
I’m not sure Node needs to publish a transpiler at all. Whatever docs page our error message links to could include a list of ones known to work, and encourage the user to choose one to install. I would think we should at least start with that approach, and if we find a need for publishing something official we could do that after MVP.
TypeScript is very slow, full of silly bugs, weird semantics. Its devs constantly-introduce complex features. Its typed system is a complete joke and doesn't take full advantage of overloading which brings performance in others languages. It also breaks backward compatibility often and simply sucks. Being made by Miscrosoft who are infamous for attempting to lock-in users and dropping their products at will (RIP Atom) is also another major issue over here. Important bugs in TS remain open for years and nobody cares about them. Choosing TS is not best choice.
We need a better typed language without all that complexity fuss.
See my post over here: https://gist.github.com/hinell/feb13b6a583f8c2a6734d8d881a45e06
There is a related proposal to add type syntax to JS, checkout:
What is the latest update on this? With both Deno and Bun improving their compatibility with Node.js and providing native support for TypeScript, there seem to be numerous examples and lessons to be learned from their advancements.
Since it hasn't been mentioned in this thread, I would like to inquire whether this proposal could enable the distribution of npm packages exclusively written in TypeScript. Such a capability would significantly enhance the user experience when working with Node.js and TypeScript, eliminating the need to juggle between .d.ts and .js files.
What is the latest update on this?
We’re exploring this space via https://github.com/nodejs/node/pull/49704 (cc @JakobJingleheimer). One idea we’ve been considering lately is that Node could possibly provide some kind of blessed or automated way to install plugins, one of which could be TypeScript. Part of the problem is that TypeScript itself doesn’t provide Node integration, and there are many third-party plugins for that purpose, and we would want to avoid picking a winner. There’s also no one correct way to set up TypeScript; it depends on your build tool and its configuration, your app and its framework and those expectations, and so on.
You might want to open an issue with the TypeScript team, if there isn’t one there already. They’re the ones who have the power to add official support for runtime transpilation and/or type-checking, and/or provide a blessed/recommended tsconfig.json
for Node projects. TypeScript itself is somewhat coupled to Node, in that the extensionless resolution mimics Node’s CommonJS resolution and the conversion of ES modules to CommonJS modules was created to address former limitations of Node; the TypeScript team is perhaps the best source for how they advise integrating their tool with Node.
I would like Node.js to support the following user experience
(I picked the typescript-node-core name at random, I would be extremely happy if that could be ts-node)
Note that
script.ts
could evaluate to either cjs or mjs depending on thetsconfig.json
file.Originally posted by @mcollina in https://github.com/nodejs/node/issues/43408#issuecomment-1182883689
Why vendoring is not an option?
tsconfig
given the 100+ options.