Additional features related to Node ESM support

alangpierce commented 2 years ago

It's no secret that ES module support in Node has been a long and challenging transition, and I'd like Sucrase to do what it can to move the process along and minimize pain points as much as possible. The Sucrase project hasn't done much explicit work so far to support ESM, and I'm hoping to change that in some upcoming work.

This issue tracks a few separate work items and lines of investigation that are all generally in the direction of better ESM support, roughly in priority order.

ESM loader for Sucrase

Currently sucrase-node and sucrase/register are CJS-only, so I think the first priority is making it easy to use Sucrase for development in a Node ESM project. For TypeScript, that means following the conventions established in TS 4.7 (particularly explicit .js extensions in imports). Currently it looks like this is a bit trickier to implement than a require hook, so I think the easiest path is to integrate with ts-node, which is happening in https://github.com/alangpierce/sucrase/issues/726 and https://github.com/alangpierce/sucrase/pull/729 . My impression is that it would still be nice to have a built-in sucrase/esm-loader that's a bit simpler and more opinionated, analogous to sucrase/register.

Prioritize Sucrase improvements that make Node ESM easier to use

Normally I try hard to keep Sucrase as low-config as possible, but I think ESM/CJS interop is probably a place to make an exception. Two recent examples were https://github.com/alangpierce/sucrase/pull/727 and https://github.com/alangpierce/sucrase/pull/728 , which hopefully will make it easy to work around issues when using an ESM-only package or migrating to ESM.

Some other ideas that come to mind:

Make sure CJS output is compatible with any special cases from https://github.com/nodejs/cjs-module-lexer so that sucrase-generated exports are recognized. https://github.com/nodejs/cjs-module-lexer/pull/53 is an example of special case support for TypeScript output.
Better config support in general, e.g. an environment variable that works with integrations that are otherwise hard to configure.
Add a CLI option for publishing a package as dual ESM/CJS.
(Maybe) Add an way to customize transpile behavior on an import-by-import basis for better interop with tricky libraries.
(Maybe) Add a transform that makes it possible to do test mocking in ESM as easily as in CJS, so that tests don't need to go through a big rewrite to switch to ESM. I alluded to this idea in https://github.com/alangpierce/sucrase/issues/715 .

If other people seeing this issue have ideas here, feel free to comment!

Use ESM for all internal scripts in Sucrase

Currently Sucrase internally uses sucrase-node for all internal scripts and transpiles everything to CommonJS. Switching all of those to use true ESM would be a good non-breaking first step before changing the package itself to fully support ESM.

Restructure package to work with Node ESM

Currently, using Sucrase from Node will always use the CJS version, which is certainly usable from ESM, but it would be nice if packages were using the ESM build directly. The package structure has an esm folder, but all imports are extensionless, so it can be used by many bundlers but not natively within Node. My plan is to restructure the package into cjs and esm folders with .js extensions for all imports, using conditional exports to point ESM and CJS usages to the right place. I believe this will be a breaking change (because of the package.json exports field, among other things), so it'll be as part of the 4.0 release.

Consider releasing an ESM-only package under an alternate name

The ESM -> CJS transform is a big source of complexity for Sucrase, and it would be great if the community eventually converged on ESM to the point where ESM -> CJS transpilation no longer feels like a necessary feature. I don't think it's reasonable to drop support for that transform as a breaking change within sucrase any time soon, but one idea could be something like a separate sucrase-esm package that's smaller and a little faster.

aleclarson commented 1 year ago

This would be really useful for the AVA test runner, which only works with ESM loaders these days (e.g. see @ava/babel here which supports AVA 3, two versions back).

DanielRios549 commented 10 months ago

Is there any update? We're almost in 2024 and Sucrase is still not working with type module in package.json...

alangpierce commented 10 months ago

Hey @DanielRios549, I believe Node ESM is already working in Sucrase, though let me know if you're running into specific issues. Sorry, I think the original issue title ("Node ESM Support") and "open" status were misleading. This issue tracks some additional plans in the same general direction as ESM support, but Sucrase should already work with just about any ESM project and use case.

I'll edit the original description with more details, but as a quick overview:

To compile-on-the-fly or use a repl, use ts-node with the Sucrase plugin. ESM loaders, interpreting tsconfig, and integrating deeply with Node all have some complexity that are the primary goal of the ts-node project, and Sucrase handles the transpiler side when you use the Sucrase ts-node plugin. As mentioned above, I've been meaning to write a loader specifically for Sucrase that's a little lighter-weight, but in the meantime, ts-node is the best way to go and will probably always be more feature-complete.
For any direct transform() use cases or integrations (webpack, etc), all necessary options that I'm aware of should be available. To target ESM, omit the imports transform in the config. To match TypeScript's module: nodenext most directly, it's best to specify injectCreateRequireForImportRequire as well. If you're working in a project with CJS code that needs to use dynamic import on an ESM project, you'll need to specify the preserveDynamicImport Sucrase option.
When using the sucrase CLI, basic cases should be supported, though I'll admit there are some gaps. The CLI can accept any of the options in the last bullet point, so that should cover everything from a standpoint of the transpile behavior. However, I see that it only attempts to transpile .ts/.tsx/.js/.jsx, not any of the newer .mts/.cts/etc. From my usage of Node ESM so far, I haven't had much use for .mjs and .cjs (and instead used the package.json type field), though I understand it can be useful for more advanced use cases. I also think it's often best to set up Sucrase compile-on-the-fly as the development experience and use regular tsc for production builds/releases, since tsc can also generate .d.ts files. That said, Sucrase does compile itself using the Sucrase CLI, and it is a use case I want to support.

If this doesn't cover your use case, I'd love to understand more details. I suppose for JSX/Flow-only use cases, ts-node isn't a reasonable option, for example. Feel free to file any additional issues or just comment here with any details about what's not working or what you need that's missing.

aleclarson commented 2 months ago

Not sure if this is the right place, but I discovered the --experimental-require-module flag in Node v22+ that allows for require of an ESM-only dependency while using Sucrase register.

https://asciinema.org/a/kmFAbZ3I53ehJKSFuxt665qok

alangpierce / sucrase