Support `.mjs` output

[demurgos]

Experimental support for ES modules just landed in Node 8.5 (changelog, PR). Since ES modules have some parsing and semantic differences, Node decided to use the mjs extension for ES modules (while js is for the "script" target and commonjs modules).

The current Typescript version (2.5.2) supports ES modules emission but uses the js extension by default. It means that to use it with Node, a post-compilation step is required to change the extension from js to mjs. This adds undesirable complexity to use native ES modules support with Node.

A solution would be to add a compiler option to output *.mjs files when emitting ES modules.

Edit (2018-03-22): The propositions below are a bit outdated. I recommend reading the issue to see the progression. See this comment for my current proposition.

Notes:

• It is possible that new parse goals will be added in the next versions of the ES spec. Letting the user provide the extension explicitly (as opposed to a boolean switch) would allow to be forward compatible with other new extension.
• It should still be possible to emit ES modules with the *.js extension, many tools rely on the js extension.
• Another solution would be to have a different extension for the source files: *.mts files would compile to *.mjs, this would be similar to *.tsx and *.jsx.

[kitsonk]

See #10939 (and #9839, #9551, #7926, #7699 and #9670) and this comment.

[demurgos]

@kitsonk Thanks for posting these link but I am not sure if they are relevant, could you explain them? I found these while checking if the issue was already opened: most of the issues you listed are about input files. This issue is about compiling ts files to mjs instead of js (adding an options to change the compiler output).

[kitsonk]

Basically those say that TypeScript does not want to concern itself with managing extensions like that, as it is already complicated enough. In particular the output extensions run afoul of the TypeScript non-goal of:

4. Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.

If you want .mjs files, it would be best to do something like:

$ npm install renamer -g
$ renamer -regex --find '\.js^' --replace '.mjs' './outDir/**/*.js'

[demurgos]

Thank you very much for clarifying your comment. I understand that adding any new feature is a burden since it means that it then must be supported for a long time, but I believe that the benefits of the mjs support are worth it: it won't turn tsc in a complex build tool but help TS users.

The long term goal of the Node team is to allow users to author their code using ES modules without paying a cost due to the commonJs modules and script target. This is one of the reasons why the proposals to require "use module" or an export statement (even an empty one) were dropped in favour of using a new extension. See the Node EPS discussions in https://github.com/nodejs/node-eps/issues/57 and https://github.com/nodejs/node-eps/pull/60. If a post-compilation step to rename the files is still required in two years only because .mjs was introduced later then it will just add another burden to remember for years (like the BOM or the different line endings)... Even if there are some discussions to support ES modules with the .js extension ("module" property), the current position is that - if such support is added - it should be used as a fallback for old tooling. .mjs is promoted for new code.

Complex build tools have their place for assets management, dead-code elimination, bundling, minification, etc. These all fall out of scope for TS, but being able to produce runnable code does not. Big projects have their own workflows and use the TS library (either directly or through plugins for other task runners), adding the renaming step is not a big deal for them. On the other hand, a sizeable chunk of projects using Typescript are small/medium sized libraries. These libraries usually only need a single tool: tsc. If they want to support native ES modules, they'll currently have to come up with a command similar to the one posted above by @kitsonk. The problems are that it raises the barrier to entry, causes duplicate effort and makes the build more expensive.

It raises the barrier to entry because newcomers can no longer simply use tsc -p && node index.mjs (--experimental-modules will no longer be required when the use-case for this issue will be relevant) and run their code to try out Typescript. For existing projects, the "build and run" command is usually already defined as an npm script or gulp/grunt/webpack/whatever task, writing this command (or understanding it) will be more difficult: for some persons it will be trivial, for others it'll require a few hours of research. The rate of change in the JS ecosystem is already pretty high, let's try to not worsen it. Now, a related problem is that it may cause some inconsistencies and duplication: different projects will use different ways to rename their file. I discovered renamer thanks to the message above, but I would have written a gulp task otherwise. Someone else would have used a POSIX-only shell command with broken edge cases or rolled their own helper Node script. To be honest, I think that most good projects would have a good implementation but this lack of standard way to deal with file renames when no build system is already there may increase the "barrier to entry" problem. Finally, once the manual renaming is configured, it still represents an additional cost. I don't have any measures but for medium projects and using Node to rename the files, it may be longer to start the VM than running the file rename: why not doing it in the same process as TS? Also: what about --watch? How do I make it play well with my custom command? Do you have to have to watch the build directory to rename as the files are emitted?

So far, most of my arguments were about the build complexity: having it done once in TS is better than having each project coming up with its own solution. There is another important reason to have ES modules working out of the box, and I think that it aligns with the goals of TS:

6. Align with current and future ECMAScript proposals.
7. Preserve runtime behavior of all JavaScript code.

ES modules are part of the spec and the TS syntax uses them. These modules have a specified runtime behaviour that cannot be fully replicated in commonJs. It affects among other things circular dependencies, early errors, mutation of exported namespace, etc. Many people won't care, until it bites them. When setting the module option to commonjs, TS does a pretty good job of generating an output with the expected behaviour but it does not trump using real ES modules. So we are in a situation where we can either set an option and have a good approximation using commonJs but if we want the exact spec and runtime behavior we have to jump through hoops and loops.

One last argument is that even if browsers support native ES modules without requiring .mjs, real-world usage of native ES modules will start server-side with Node because you can control its version. Node is important for the whole ecosystem so it would be damageable to just ignore that it uses .mjs for ES modules. You can't just treat it as a "platform detail" when this platform is used by the majority of the TS audience.

To summarize, here are my main points:

• Now that native support exists for ES modules, people will want to use it out-of-the-box.
• Node settled on .mjs and hopes that it'll gain traction. .mjs is Node-specific, but Node is important
• Many libraries only need tsc, requiring a second tool for renaming increases barrier to entry and causes duplicated effort (of varying quality).
• Using a post-compilation command leads to higher build times and does not play well with the watch mode.
• Using native ES modules is the only way to match the spec exactly, TS should not make it hard to use

[SMotaal]

Just an Opinion

As a very strong fan of TypeScript, and obviously a user of node (I guess even TypeScript is) I would really like to see some agreement between them on an issue that has had so much disagreements over many years of rational and sometimes awkward discussions.

I would have loved it if node had a transition phase to make common js (the non-standard) become .cjs over a period of two years.

That said, mjs seems to be the current direction, obviously that non-js file extension (either one) itself does not matter, but the ability for TypeScript to provide a way to control this inline as the originator of the transpiled files is something that really falls to TypeScript.

That said (again), mjs as a locked extension is kind of very opinionated, someone must have realized that forcing js was okay, it was already called js, but forcing mjs, come on, even my spelling checker is nagging me on that one. I hope nodejs will become a little more flexible on that one.

Proposal

Please make it possible to specify mjs as a module format that simply means compiling es2015 modules and calling them mjs :)

[trxcllnt]

It's important to note here that .mjs isn't only about file extensions. running node with the --experimental-modules flag also means that ES6 module specifiers must be valid urls, so the extension has to be included in the import statement, e.g. import x from './x.mjs

If I'm understanding this right, renaming the files isn't a solution, TS would also need to compile module specifiers with the .mjs extension in the ES2015+/mjs mode. We're keen to support newer versions of node by publishing non-downleveled iterators as es2015+ modules for IxJS, so it would be great to get an answer on this soon.

[dpogue]

TypeScript already allows you to include a .js extension on your imports (even when you're actually importing .ts TypeScript files). The change needed would be to expand that to also allow .mjs.

[trxcllnt]

It's not about the source files including the extension, it's about the compiled files.

If I compile import x from './x' into ES5/CommonJS, TS emits:

var x = require("./x");

If I compile it to ES2015/ESModules with an .mjs extension, TS should emit:

import x from "./x.mjs"

If a library is trying to support both old and new node with CommonJS and ESM side-by-side (as *.js and *.mjs extensions respectively), node only imports the .mjs files if the extension is included in the module specifier.

[SMotaal]

There are some potential updates in the works… including a --loader option that would you to specify a file with a resolve hook that will take the specifier and parent module path, then it could handle extension prioritization as needed. It seems to be very flexible but their goal is to make it declarative and not have people overloading the actual loader functions.

In terms of picking mjs over js, that is only the case in the CJS loader system which uses the now legacy Module prototype with it's hooks, the ones that everyone likes override all the time.

In the current --experimental-modules release, the ESM loader is hard coded to '.mjs' or falls back to the CJS loader.

What I've seen so far over the past few days makes me believe that the ESM loading system will take over with pluggable CJS loading (unless opting to use the legacy-modules by flag or based on the main file).

Check out this PR/discussion

[SMotaal]

@trxcllnt I guess regarding the part about baking the extension right into the import… this applies to import/export everywhere now as part of loader specs.

When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.

So if TypeScript would continue to work without third-party tooling, it will need to find a strategy to write out standard (not ISO but platform-specific) out-of-the-"compiler"-box projects that will just work in either node or browsers at least depending on the compilerOptions intent specified by the user, and do so without asking the user to mockup some hack to get it to work.

This is just an opinion, but honestly, it feels like it for TypeScript to address.

Checkout this MDN reference but make sure you notice the part where it first said "excluding the .js extension" then in the examples included the extension anyway.

[bmeck]

I'd like to include here that the IANA has an Internet Draft which specifically adds .mjs for COMMON usage that represents the Module goal of ECMAScript . .mjs is not purely a Node.js concern and is even included in an example for browser specs and is supported by a variety MIME DBs already like shared-mime-info.

@daflair can you expand on what

(unless opting to use the legacy-modules by flag or based on the main file).

means?

The next step after that PR is to setup per package loaders to be able to guard against global loader mutation. So, it might be going the direction you think.

When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.

Interestingly the WHATWG Loader Spec had a very early version in the ECMAScript spec that was removed at the last minute before ES2015! There are interesting other things like the original CommonJS Spec which mandated not to have extensions. We should probably avoid dwelling on the past so much since the different loaders vary so much on these opinions.

Node's EP specced a superset of the WHATWG resolve algorithm that does do various Node idioms like file extension completion. However, the browser has a subset that is safe to use in Node. Still, even with that subset there are problems with dependency trees since "bare" imports are waiting on userland feedback (you can get involved in the hook here), but is mostly left up to intelligent servers and service workers for now.

I'd recommend trying to compile down to the WHATWG compatible specifiers except for bare specifiers for now.

[SMotaal]

@bmeck what I meant by:

unless opting to use the legacy-modules by flag or based on the main file

I was making an assumption that at some point you will have to use something like --disable-es-modules or --legacy-modules which is really an option that simply does the opposite of --experimental-modules (when this flag is the default behaviour) to resort the current loader system and completely bypass anything related to the new loader (ie legacy applications that simply find ways to be incompatible with this existential change to their eco system).

But now that I dug in a little deeper in your recent PR's this might not be your intent…

So it is best to ask you about your intent here 😉?

[bmeck]

@daflair I'm still not understanding. ESM support in --experimental-modules completely avoids touching CJS except by overtaking the defaulting of .mjs to CJS (it now throws). I can't think of a reason to introduce such a flag. Also, it would be a very hard sell to change the default behavior from CJS to ESM or vice versa and I doubt that will ever happen.

[SMotaal]

Okey, when we get to the point where --experimental-modules is no longer needed, at that point, will the CJS loading system that exists today still remain (mostly) unchanged?

If so, at that point, without the above flag and without any other flags like --loader, …etc :

a. Would running with a .mjs entry do what it does today with the flag and the CJS loader will simply pass it to the ESM loader? meaning that all dependencies will be marshalled by the ESM loader (even if it delegates something to the CJS loader).

b. Would running with a .js entry always imply CJS (again no special flags) and this also means that ESM loader is essentially completely idle for the lifespan of the process (assuming there will be no require() support for es-modules in the future).

[bmeck]

Okey, when we get to the point where --experimental-modules is no longer needed, at that point, will the CJS loading system that exists today still remain (mostly) unchanged?

Yes, it is unchanged except the .mjs reservation.

a. Would running with a .mjs entry do what it does today with the flag and the CJS loader will simply pass it to the ESM loader? meaning that all dependencies will be marshalled by the ESM loader (even if it delegates something to the CJS loader).

No, the 2 loaders are decoupled except when ESM defers to CJS. Loading via import will always go through the --loader hooks. Loading via require will not go through the --loader hooks. Use import for green code going forward.

b. Would running with a .js entry always imply CJS (again no special flags) and this also means that ESM loader is essentially completely idle for the lifespan of the process (assuming there will be no require() support for es-modules in the future).

Yes .js is CJS. However, you can use import() to get a hold of ESM since that is available in all JS (even eval).

[kitsonk]

And people wonder why the TypeScript team wants to avoid getting into this area at the moment... 🙄 No it is, yes it isn't, no it is Even if TypeScript gets further into an extension mangling business it would wait until it was clear that --experimental would be retired as a flag and established as a default with appropriate battle hardened semantics.

[bmeck]

@kitsonk

🙄 No it is, yes it isn't, no it is

can you clarify what is unclear in the messaging from Node?

[trxcllnt]

@bmeck so one thing that seems like a major foot-gun is how node doesn't automatically select the mjs files when importing a module with --experimental-modules mode on. I understand the story on importing ESModules from the node_modules folder isn't complete yet (should it look for the mjs file path from "module" in package.json? who knows). But automatically selecting the CommonJS form of a module then converting its module.exports object to the default export means that code that works fine in other ESM environments is broken in node:

// node_modules/ix/Ix.mjs (no default export)
export class Iterable {};

// node_modules/ix/Ix.js:
module.exports = function Iterable() {};

// run with node --experimental-modules some_file.mjs
// works fine
import { default as Ix, Iterable } from 'ix/Ix.mjs';
assert(typeof Ix == 'undefined') // true, great
assert(typeof Iterable == 'function') // true, great
// broken
import { default as Ix, Iterable } from 'ix';
assert(typeof Ix == 'undefined') // false?
assert(typeof Iterable == 'function') // false :(

[bmeck]

@trxcllnt it is complete and has been laid out in depth in https://github.com/nodejs/node-eps/blob/master/002-es-modules.md

should it look for the mjs file path from "module" in package.json?

no.

But automatically selecting the CommonJS form of a module then converting its module.exports object to the default export means that code code that works fine in other ESM environments is broken in node

Correct, because this code cannot be spec compliant. There is absolutely no way to link it ahead of time since you need to eval to get the shape of module.exports. Emulations using CJS semantics like babel or typescript are not enforcing the shape of a module during Instantiate.

This style of eval based shape detection will never be possible.

In the future there may be a pragma that allows parse time instead of eval time declaration of shape, but that would still be a breaking change.

Simply put:

works fine in other ESM environments is broken in node

Is only true because those environments are not valid ESM

[trxcllnt]

@bmeck I'm afraid I wasn't clear. I'm not saying node shouldn't use the exports from a CommonJS module as the default when --experimental-modules is on.

I'm saying that when I run with --experimental-modules and I import a module from node_modules that has both a CommonJS and ESModules form side-by-side, I would expect node to select the ESModule over the CommonJS one.

I have this expectation because I'm explicitly running node in ESModules mode, and using an ESModule import statement to import a package that exports an ESModule. But instead because node selects the CommonJS version (and then exports becomes default export), the code doesn't do what I expected it to do.

And to clarify why this matters, we're compiling es2015+ features (generators, async generators) down to ES5/CommonJS as the .js files, and compiling the es2015/ESModules as the .mjs files. The ES5 files have all the iterator downleveling codegen necessary for ES5, but ideally we can run node with --experimental-modules and use the native-generator version of the lib automatically.

[bmeck]

@trxcllnt it does search for .mjs first. https://github.com/nodejs/node/blob/e00a488731ed333c3cd0952acdfe85729b734fa5/src/module_wrap.cc#L37

Your specifiers are different in meaning and you can see if you use import from 'ix/Ix'; that .mjs is preferred. I assume your package.json is explicitly pointing to the .js but was unsure given the example not providing one.

[trxcllnt]

@bmeck npm install ix to check it out for yourself, but the relevant bits of the package.json are this:

{
  "name": "ix",
  "main": "Ix.js",
  "module": "Ix.mjs"
}

We can't set "main": "Ix.mjs", as that'll throw when not using --experimental-modules. Right now we set "module" to the mjs file because most of the major bundlers (webpack, rollup, etc.) will use the "module" path over "main". But from everything I've read so far, there isn't an ESM analog to CJS's "main" entry in package.json (please correct me if I'm under-informed on this).

[trxcllnt]

@bmeck and also I want to clarify, you and I are on the same side of this issue. .mjs is just fine with me, as long as things have reasonably normal default behavior. I'm in this thread arguing that TS should generate imports with .mjs extensions, because it seems all ESModule import specifiers are now required per the spec to be valid URLs.

Considering the prevalence of importing libraries from node_modules in node, I was a bit surprised there wasn't clarity on this issue for ESModules. And also that it also did a bad/unexpected thing when we're explicitly trying to support both CJS and ESM in the IxJS project.

I would even have preferred node to throw/print some sort of "ambiguous import" warning like "hey dummy, you tried to get an ESModule from a package.json file which doesn't have a way of specifying ESModule mains, here's the ES5 version but maybe be more explicit next time"

[bmeck]

Remove the extension from "main". Just "Ix"

On Oct 3, 2017 10:00 PM, "Paul Taylor" notifications@github.com wrote:

@bmeck https://github.com/bmeck and also I want to clarify, you and I are on the same side of this issue. .mjs is just fine with me, as long as things have reasonably normal default behavior. I'm in this thread arguing that TS should generate imports with .mjs extensions, because it seems all ESModule import specifiers are now spec'd to be valid URLs.

Considering the prevalence of importing things from node_modules in node, I was a bit surprised there wasn't clarity on this issue for ESModules, and it also did a bad/unexpected thing when we're explicitly trying to support both CJS and ESM.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Microsoft/TypeScript/issues/18442#issuecomment-334037289, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOUo4SRH5PQaknAohmUHU330-YDy7-mks5sovTkgaJpZM4PWbkF .

[trxcllnt]

@bmeck HMMM yeah that sounds like a solution. sorry for hijacking the thread everybody.

[SMotaal]

Awesome @trxcllnt and @bmeck, I guess we are all really trying to figure things out, so a resolution is always a great addition to a thread.

By the way, many of us nagging at Node to support ESM with .js are also nagging at TypeScript to support "module": "mjs" that can run with node directly without a --loader. Different workflows, different requirements.

So here is a silver lining:

// hybrid package.json
{
  …
  "main": "index",
  "files": [
    "index.js",
    "index.mjs",
    "index.d.ts"
    "index.es.or.whatever"
  ],
  …
}

Then if I do my work right, I would really appreciate being able to add this:

// hybrid package.json
{
  …
  "scripts": {
    "compile-cjs": "tsc -m commonjs",
    "compile-mjs": "tsc -m mjs",
    "compile-whatever": "boink plug and play box yes no okay --but-follow-es-standards-please",
    "compile-all": "npm-run-all compile-cjs compile-mjs …"
  },
  …
}

Instead of having to do this:

…
    "compile-mjs": "tsc; rename …; find-replace … from /.*?/ with $1.mjs --verbose"
…

Does that sound about right?

@bmeck so those index variants will resolve to mjs if imported and js (cjs) if required when things are stable without any special flags, right? @demurgos is this what you had in mind? @trxcllnt is that in line with your conclusions so far? @kitsonk are we any closer to convincing you?

👍 Let's keep at it!

[kitsonk]

are we any closer to convincing you?

Convincing me is irrelevant. I am just a vocal member of the community, with their own opinions.

[bmeck]

@daflair

so those index variants will resolve to mjs if imported and js (cjs) if required when things are stable without any special flags, right?

correct. "main" follows normal resolution. It would check all the file extensions in order with .mjs first.

[trxcllnt]

@daflair yes this worked great, and is now out in ix@2.1.4. We have an interesting packaging strategy already in place to accommodate the wide spectrum of node/browser/electron bundling needs and support for Iterable/AsyncIterable (some people can target newer runtimes, others want to down-level), so it's not hard to rename the es2015/ESM files to .mjs in the build process. Considering we follow a similar strategy in Arrow, I've been wondering if this deserves to live in a CLI package so other folks can use it, ala create-react-app (hmu if you're interested in helping w/ that!)

[styfle]

It would check all the file extensions in order with .mjs first.

This works for me! Thanks! 👍

I was able to get my project, copee, to emit CJS and MJS. It works in Node and in the browser!

I also wrote about my findings in this blog post if anyone wants to do the same.

---

One thing that I should bring to the TypeScript team's attention is https://github.com/babel/babel/pull/6455 since it is dealing with babel emitting a pragma and therefore would be relevant for TypeScript.

[demurgos]

Thank you for your post. I had some time lately to properly experiment with the current mjs implementation. I'll try to write a better summary this week-end. So far I have 2 main issues related to interoperability with commonJs modules (so core libs and most of npm).

I want to have the same .ts file compiling to both .mjs with ESM and .js with cjs. Commonjs modules are wrapped as the single default export for their ESM namespace. As a result, when I import a .js file from .mjs, the semantics of the cjs module are similar to export const default = module.exports;. This kind of wrapping is similar to what SystemJS did and the reason why allowSyntheticDefaultImports was introduced. When enabling this option, I can use import fs from "fs" for .mjs to get the semantics of import * as fs from "fs" in .js, but it's broken in .js ("commonjs") because all the references to fs are replaced by fs.default. I haven't found yet how to get "dual" builds only by tweaking the compiler options.

A second issue is that synthectic default imports are triggered only when the module does not have a default property already. This breaks cjs modules that actually have a default export. For example exports.default = "foo"; in a cjs module gets exposed to mjs as {default: {default: "foo"}} and import foo from "./foo" returns {default: "foo"} but Typescript infers only "foo".

The scenario for projects without any direct dependency on cjs modules will work great once the emission of mjs extension will be supported. Proper interrop with cjs may require more work.

[SMotaal]

Thank you @demurgos and @styfle for the very insightful findings. I've also been exploring interop aspects between c/mjs literally (yes, I opted for vanilla in my newest side project) to if I can write interchangeable code completely separated from linkage (imports/exports) instead of trying to using today's TypeScript to transpile across node's flavours — so focusing more on better practices to decouple functional code from linkage.

The side project was a single file which I wrote as CJS & MJS simultanously making sure that both files executed similarly as I worked on them. I might have cheated a little because it required node modules only (zero-dependencies) and did not even require local linkage since it is one file. This scope made it ideal for a pattern to evolve and I quickly went from two similar implementations to identical code (beyond linkage).

So I can confirm that destructuring which is necessary for node's import fs from 'fs'; is functionally indifferent when working with const fs = require('fs');, so that is a great plus for node for opting for default-wrapped exports. The downside of course is that destructuring defies the better practices for promoting static analysis (I guess those tools will still having to carry some of that burden they so cleverly avoided after all). The good news is that TypeScript's language service was able to cascade equally across both files and no inconsistencies stood out, so thats a relief.

This pattern worked for my side project to the point that I eventually could literally copy/paste the entire body of the file (beyond the first few lines of linkage) across both .js and .mjs files — at least for this very limited scope.

A key take away is that until issues like the yet-to-be-determined fates of module-scoped fields like module, __filename__, and the likes, and until dynamic imports are locked-in, it's logical for TypeScript's team to hold off on devising a module transformation.

Still, I hope once those aspects are resolved, TypeScript (and all other transpilers out there) really buy into supporting the awkwardly received "MJS" format.

Check the code here: https://gist.github.com/daflair/638b82842f6d083dbb41607ad737badf

[demurgos]

Regarding interop with commonjs, I guess that https://github.com/Microsoft/TypeScript/pull/19675 is worth following. It should allow to get import fs from "fs" to work when compiling both for cjs and es2015.

[SMotaal]

On that topic... I found two possible solutions:

1. In tsconfig:allowSyntheticDefaultImports used with module ES2015, then back-transpile to CommonJS using rollup (my work is ESM centric so I didn't fuss with CJSing the output but ESM works)

2. Using a loader with dynamicInstantiate: https://github.com/nodejs/node/blob/d21a11dc23d6104b1d03fa2ddc1c808dcaf89c31/test/fixtures/es-module-loaders/builtin-named-exports-loader.mjs

On the dynamicInstantiate part, I hear the node team might be pulling the plug on that part, but so far it's been more than reliable, and I did make it a point to share my thoughts on keeping it alive for what its worth.

[tommitytom]

I'm a little confused that there appears to be no method of achieving this.. so I'd just like to clarify what the options are for my use case before I delve down this rabbit hole:

I want to use ES modules in node and the browser the way they are intended to be used. No CJS, no extra build tools or transpilation other than tsc.

I would currently import a module in the following manner: import Logger from './logger';

For this to be a valid ES module import, the .mjs extension needs to be added.
import Logger from './logger.mjs';

But tsc does not like this. Am I correct in thinking there is no way to get this to compile as expected? From what I can tell the only option I currently have is to:

• Use the old method of importing modules
• Rename all the .js files that tsc outputs to .mjs
• Run a find and replace on all of these files and add .mjs to any import statement that is found

If that is indeed the case I hope this issue gets resolved swiftly!

[trxcllnt]

@tommitytom if you're targeting ESM in node, you don't have to change anything about the import declarations in your files. Just change all the extensions from .js to .mjs after TS transpiles them and (assuming your entry point is also an .mjs file) node will import .mjs files when it encounters an ESM import declaration.

[demurgos]

@tommitytom and others

Since I know it's not easy to get TS and mjs running with all the moving parts, I made an example repo: https://github.com/demurgos/mjts

It demonstrates how to compile the same source code to both commonjs and esm and how to import commonjs or mjs (in this case even hybrid) modules. Note that it uses the experimental ESMInterop option that is not yet merged (https://github.com/Microsoft/TypeScript/pull/19675) so you'll have to do a recursive clone.

This module does not use hybrid modules in the same package but you can do it trivially by emitting to the same output directory. @tommitytom As you can see, you should not use extensions in your import declarations. The only case when you might needed it is if you have both foo.js and foo.mjs in the same directory and, from an ESM, want to forcefully load the commonjs foo.js instead of the ESM foo.mjs (I use it to get a hold on __dirname or create ESM interfaces).

[tommitytom]

Thanks for the swift replies! Where this is fine for node, this will cause issues in the browser, right? Since from everything I've read, and from my owns tests, you have to supply an extension (js/mjs/whatever) when using ESM modules in the browser. And in my case I have a massive amount of code that is shared between 2 projects, one which is node based and one which is browser based.

[demurgos]

I have to retrieve the source but I've read that the recommended way to work around this is to treat the extension as an implementation detail (like .php or .aspx) and configure your server to server to reply to /scripts/my-lib with ./scripts/my-lib.mjs.

This can be done fairly easily if you can control the server but does not work well when you don't (Github pages?). In this case, the reply was to use some build tools but I agree that it is not always ideal. I'm also interested in the possibility to have the extension in the output file. Even if this is useful for .mjs, it is also kinda independent. You can maybe track this issue if you're interested in this.

Edit: Here is the discussion I was referring to: https://github.com/nodejs/node-eps/issues/51#issuecomment-284006041

[tommitytom]

I have control of the server but I'm constantly using different web servers with this project - including hot reloaders and stuff and don't really want to go down that route as it's way harder to maintain and document for other devs.

I ended up writing a small script that does the find-and-replace as I described above and have my project working in both node and the browser with pure ES modules :)

And I must admit it feels GREAT - it's made me very excited about the future of JS, and getting rid of all the awkward build scripts that we've all become accustomed to with gulp/webpack/grunt etc has been such a breath of fresh air :)

The only issues I'm facing right now are getting dependencies working well in code that is shared between the browser and node - I have a hacky work around for now but I'm sure this will be solved in the future when other people reach these issues

[demurgos]

Synthesized namespaces got merged 🎉

This solves the issue with consuming CommonJS modules from Node's native ESM. You can now easily have a single .ts file and emit code working with both {"module": "commonjs"} and {"module": "es2017"}. You have to opt-in to this behavior using {"allowSyntheticDefaultImports": true, "esModuleInterop": true}.

The main change is that you can only use default imports for commonjs: this aligns it with Node's behavior that is unlikely to support named exports from commonjs. You can still import types as named exports but I would not recommend to mix default imports and named imports.

// Code you probably used previously, this does not work if you want both ES and cjs
import {parse, Url} from "url";
const result: Url = parse("https://github.com");
// Instead you need to use
import url from "url";
const result: url.Url = url.parse("https://github.com");
// Since `Url` is an interface and does not affect the runtime, you could do the following:
// import url, {Url} from "url"
// const result: Url = url.parse("https://github.com");
// (But I don't recommend it because it's prone to errors: you can only import named types)

The merged PR allows a good amount of code to be published for ESM. There is still an issue if you rely on some of the module meta variables such as __filename or __dirname. This is solved in v8 8.4 using import.meta. Node 9 uses v8 8.2 so I guess that we'll have to wait for Node 10. This is not a blocking issue because there is a workaround. You can have a module that is only imported as cjs and re-exports the variable you need. Here is a real world example where I have meta.ts until import.meta lands:

export const dirname: string = __dirname;

export default {dirname};

meta.ts will be only used as cjs, it will export something like this:

module.exports = {
  default: {dirname: __dirname},
  dirname: __dirname,
  __esModule: true,
}

I could not find a solution that does not repeat the exported members. The default export is used by ESM while the whole object is used by cjs.

To use it, I have a file main.ts:

import meta from "./meta.js" // Note the explicit extension
console.log(meta.dirname);

The use of the explicit extension forces Node to use the cjs version (instead of .mjs) so I can just build all my files for both cjs (.js) and (.mjs) without having to do some conditional file exclusion. (I use Gulp with this code if you're interested).

This generates the following output:

- main.js
- main.mjs
- meta.js
- meta.mjs // Unused

And you can successfully run both main.mjs and main.js.

The longer term solution involving import.meta will most likely require some Typescript changes if you want to support older Node versions. It will require some transpilation to replace import.meta.url by the correct equivalent using __file (same thing for import.meta.isMain).

The release of ESM interop is still bleeding edge. I'm experimenting with it and encourage you to do the same thing but expect some tools to break. Apart from the IDE not recognizing it yet or an already fixed error caused by Tslint, my main issue is that Typedoc is no longer able to generate documentation. I also had some issues when importing packages that were compiled without the options discussed above: it is most likely because I publish the .ts sources along the .js and .mjs source code and I have misconfigured typescript to prefer .d.ts instead of .ts. Please report any issue you found with it.

Finally, I'd just like to discuss the .mjs extension again. By default Typescript only emits .js files and I use Gulp to rename them to .mjs. Support by the compiler would greatly improve UX. I also quite like my setup that lets me select between OutModules.Js .Mjs or .Both. My .Both setting generates .mjs alongside .js with a single reporter instance so I don't get duplicated error messages. I'm not sure if Typescript want to go this far, but it helps me to publish my packages to npm with both module types (and have an escape hatch from ESM to cjs for cases like __dirname as described above).

[manisto]

This doesn't solve the initial issue all the way up top, which explicitly states that it is undesirable to have to use an external tool (like Gulp) to rename files. This solution seems overly complex if I simply want to code my own Node.JS project entirely with EcmaScript Modules.

[demurgos]

@manisto Please read the first messages: I explicitly stated that I am against requiring external tools: this is the point of this issue. My previous message is a summary of the current situation: what is currently possible with native TS, what is missing and possible workarounds. It helps people who are willing to try to use .mjs with Typescript and Node today while it is experimental. ESM support in Node is getting stable but things are still moving. Just a few days ago npm posted (yet another) proposal. Ultimately I believe that .mjs will gain first class support in TS but starting to test it with real projects will help to draft a proposal.

[demurgos]

It's been a few weeks: it's time for an update.

First a small info to get it out of the way: Node is working on providing ESM variants of the built-in libs. I don't think there should be any big issue with it but it's nice to have.

Now, the big thing currently is the "mode": "esm" PR. The goal of this PR is to make Node aware of package scopes and treat .js files as ESM or CJS depending on the value of the "mode" property in package.json. It means that adding "mode": "esm" would allow to execute .js files as ESM, so you would no longer always need .mjs for ESM. This flag affects all the files in the package scope: this means that this method does not support mixing cjs with esm. According to the discussion, the main use-case of .mjs would exactly be to support both modes in the same package. As such, .mjs would move to be a compatibility feature. Top-level application would not need it at all if they wish so, but libraries may still want to use .mjs to provide both cjs compat and good integration with esm.

---

If this Node PR gets merged, I think that a good way forward for TS would be to add support for .mjs as an "mjsModule" compiler option with the same possible values as "module". It would need to be properly defined, but my general idea would be to have the following behavior:

• "compilerOptions": {
  "module": "commonjs"
  }

  Produces foo.js files with cjs syntax. (this is how it already works today)

• "compilerOptions": {
  "module": "es2017"
  }

  Produces foo.js files with ESM syntax with dynamic imports. (this is how it already works today)

• "compilerOptions": {
  "mjsModule": "es2017"
  }

  Produces foo.mjs files with ESM syntax with dynamic imports. (no .js)

• "compilerOptions": {
  "module": "commonjs"
  "mjsModule": "es2017"
  }

  Produces both foo.js with commonjs syntax and foo.mjs files with ESM syntax (and dynamic imports). It would support first class error deduplication, watch mode, sourcemaps etc.

All the other compiler options are shared. The mjsModule compiler option should have the same end-result as compiling with module using its value and then renaming to .mjs. The fact that all the other options are shared is intended: Node moves to view .mjs as a compatibility mechanism for the module type/parse goal. I don't think larger changes between .js and .mjs should be encouraged, but you'd still have the option to use two tsconfig.json files (tsc -p tsconfig.cjs.json && tsc -p tsconfig.esm.json) with the trade-off that error messages may get duplicated.

[thetutlage]

@demurgos On your last comment.

This still leaves the custom loader hooks behind.

[demurgos]

@thetutlage Could you expand on loader hooks and how you'd like to use them here? I feel that Node is currently doing a good job with them but that they are unrelated to this issue (TS already supports ES modules but not the .mjs extension). I see them more as a feature that could be used by tools such as ts-node.

[thetutlage]

@demurgos You are right, maybe it's for project like ts-node ( not 100% sure ). So here's my use case.

For framework like adonisjs.com, we have a custom dependency resolver, which does couple of more things over CommonJs to resolve a dependency. Till now, it was not part of any standard, since Javascript has no standards around module loading.

With the new loader hooks (part of the spec), we will simply remove all the custom syntax and use that hook instead.

Now everything will work great in Node.js, but I cannot neglect Typescript and it's eco-system, as I know Vscode uses Typescript for intellisense, it will fail, since the logic to resolve a module is not the default one but instead provided via hook.

For example:

import User from 'Models/User'

Now Models can be a directory in your project root, or it can a dynamically registered model by some package. At runtime, when Node will use our hook, we will return the right path, but for intellisense this file doesn't exists.

Another use case is of webpack aliases. They also hack upon the require method, but I can see them also using a cleaner approach (loader hooks) in the near future.

1. I know you can define paths for webpack aliases today also, but aliases is a very simply use case. Whereas, loader opens up a whole new world of possibilities.
2. I know Typescript doesn't support anything I explained so far, but I am more curios to know, if they will consider this in the near future.
3. Also happy to share insights and provide coding help, if I know there is/will some work going on same.

[demurgos]

@thetutlage Sorry for the long delay. Loader hooks, custom resolvers and other mechanisms for dynamic modules are important but I don't think they are specific to the .mjs extension. Bundlers such as webpack already let you do some crazy things with imports since a long time. Still, I see your point that having a standard way to handle this may make custom resolvers more popular and gives TS a place to start supporting these behaviors. The problem with these custom resolvers is that they are basically code: I don't think you can describe their behavior declaratively (to pass it as a compiler option). A solution would be for you to write a build task to dynamically generate the compiler options or declarations for your dynamic modules. This is an approach I use personally: my compiler options are generated and passed directly to the compiler, I just emit tsconfig.json files so my editors understand what's going on. It works fine, but also requires continuous maintenance as things evolve. You may also want take a closer look into the compiler API, but the more complex your build process is, the harder it will be to make it play well with editors. (BTW: not just editors, but also collaborators: they need to understand what's going on).

I recommend you to open a dedicated issue to track this problem.

[jpike88]

Looks like things are stalling again... such a shame, ES6 modules have been incredible for my codebase and it sucks that I can't take advantage of TS to lint it, at the very least.

[jpike88]

Anyone know alternatives for linting? Using VSCode ideally.