.mjs extension trade-offs revision

[YurySolovyov]

I obviously can't speak for the whole community, but it seems like a lot of people are not happy with .mjs.

One of the main arguments to keep .js is that if we can detect 99% of cases where we CAN tell if is it CJS or ESM (or where we just know what to do), we may just call rest 1% edge cases and deal with it.

We can even come up with some linter rules and/or workarounds to simply teach people to do the right thing.

[WebReflection]

It just uses a different non-standard convention.

which is what NodeJS does already. If that's inevitable, can we at least make such convention not breaking everywhere else?

'cause import any from './module'; is a very ambiguous, breaking, unreliable, pattern easily destroyed by transpilers and import any from './module.mjs'; is not de-facto portable/standard as import any from './module.m.js'; would be.

[vkurchatkin]

which is what NodeJS does already.

Yes, but Node uses simple and cheap convention. @std/esm is pretty much experimental and not made for production, so it can do things like this.

[WebReflection]

fair enough, but beside @std/esm, my proposed convention is also cheap like the current one.

I've also done some basic benchmark and the performance cost seems to be irrelevant.

[ljharb]

node uses require.extensions, which only looks at the extension - in foo.m.js, .js is the extension. It'd have to do additional parsing of the base filename to locate the .m - and if anyone in the entire ecosystem already had an .m.js file, it would suddenly break as a result. Whereas, anyone with an .mjs file would suddenly start being able to require it.

[WebReflection]

if anyone in the entire ecosystem already had an .m.js file, it would suddenly break as a result.

so let's break 80% of the server-side world currently incompatible with .mjs instead?

Whereas, anyone with an .mjs file would suddenly start being able to require it.

you are not understanding the issue related to .mjs

NodeJS people are not using .mjs extension, they are omitting it, causing potentials disasters.

Nobody wants to use .mjs and indeed everyone is omitting it. Doesn't that tell you something?

Also, from a 100% valid ESM module, you should be able to load a valid ESM module, which is not the case in NodeJS.

So, from an ESM module, you need to know upfront what you are importing, omitting the extension, without any guard that the imported value is the expected, instead of a transpiled one.

I wish you would read my blog post that show real-world issues with this decision, and the real-world consequences for modules authors.

[Bamieh]

@WebReflection can you link your blog post?

I agree with you that i do not see anyone jumping over to use .mjs for their imports (personally I'd rather use babel). Actually i havent seen 1 .mjs file so far.

[WebReflection]

@Bamieh https://codeburst.io/the-javascript-modules-limbo-585eedbb182e

[demurgos]

Hi, I'd like to chime in and answer some of the points you raised in your blog post and comments. I hope that it will help to clarify the situation.

• NodeJS people are not using .mjs extension, they are omitting it, causing potentials disasters.

  A more pragmatic explanation is that Node support for ES modules just landed as an experimental feature a few weeks ago. You can't blame people for not switching to ES modules for Node overnight when it's not even stable.

• We now have a non standard behavior that works only in NodeJS

  Which standard are you referring to? The resolution logic is implementation-dependent according to the ECMA spec:

  The actual mapping semantic is implementation-defined but typically a normalization process is applied to specifier as part of the mapping process. A typical normalization process would include actions such as alphabetic case folding and expansion of relative and abbreviated path specifiers.

  The host is free to apply any arbitrary mapping to get the source code for a given (referencingModule, specifier) pair. This is reflected in the WHATWG loader spec or the experimental Node loader pipeline for ES modules (it just got merged). As an example, a loader based on the "module" property was implemented. Basically, the spec leaves the semantics of the module specifier to the host.

• That breaks because our transpiled module.mjs uses a CommonJS+Babel convention with an exports.default property that will not be handled by NodeJS magic importer while importing.

  I am not sure I understand what you describe. If your package exports module.mjs it should be an ES module (not a transpiled file using CommonJS+Babel). It was already discussed but the "Babel convention" is a best-effort attempt to mimic the standard but it is impossible to support the ES standard only with Node's commonjs due to edge cases such as circular dependencies, early errors and the order of resolution/evaluation (declarative VS imperative). It needs native support because the semantics are not the same.

  If you are a package author and want to distribute your package both as an ES module or commonJs module, simply provide two entry points:

  package.json

  {
  "name": "my-hybrid-package",
  "main": "./module",
  "module": "./module.mjs"
  }

  Do not specify the extension for main so Node picks the appropriate version. If you want to be extra-sure that Rollup and Webpack will pick the ES version, you can add a "module" property.

  module.mjs

  // ES entry point
  export function foo() {
  console.log("Hello from ES");
  }

  module.js

  // cjs entry point
  exports.foo = function () {
  console.log("Hello from cjs");
  }

  In your main project, you can now use import {foo} from "my-hybrid-package". Of course the goal is not to manually maintain implementations in both files. You can either author your code using ES modules and then use a build tool to generate the cjs version (you seem to already have Babel in your pipeline) or just keep your cjs code and add a single mjs file that reexports your main cjs module using the declarative ES syntax.

• so let's break 80% of the server-side world currently incompatible with .mjs instead?

  Node will not drop support for commonjs. Once ES modules and .mjs are stable, upgrading Node won't break your cjs code. Do you have an example of breakage introduced to existing code caused by the support of mjs?

• Finally, I am not quite sure I understand the rush for server support of mjs: it will come pretty quickly anyway. There is a pending request to add mjs to the IANA mime registry. Are you serving browser ES modules in production using a static server you do not control, with the limitation that the files must be isomorphic and that you cannot run any build step before pushing your files? If so, you're on the very bleeding edge and have to expect some issues. Even after reading your blog post and comments, I don't see your real-world use case that is impossible due to Node using mjs for ES modules. Could you give a more concrete example?

[WebReflection]

I cannot publish in npm a transpiled module or it will break if an ESM will import it omitting the name.

I'm not on the bleeding edge of anything, check this post too: https://github.com/nodejs/node/pull/16170#issuecomment-336318420

I have 200 repositories and I'm the least person that uses tools. Babel is never on my way except when it comes to interoperability with other projects, now compromised in core for various reasons.

However, I didn't know the loader landed and I didn't know it could've forced ESM in non .mjs files, as suggested in here so that might be the solution to, at least my problems.

On top of that, I wonder if the "module" entry in package.json is, or will be, compatible with regular .js files instead of .mjs.

If that's the case and it's landing in Node then primitives to have full ES2015 env based on .js files and cross platform is possible, making both my proposal, and .mjs (IMO) useless.

Thanks for providing extra details.

[WebReflection]

@demurgos am I correct expecting that this loader would enable my proposal and will work via node --loader es.mjs ?

I have errors for bad option right now, thanks.

[ljharb]

The (nonstandard) "module" field is only used by two bundlers; node will ignore it - I believe neither bundler cares what the file extension is for that field.

You can publish in npm a transpiled module (transpiled into CJS, presumable), with the .js file extension, and it will never break due to ESM support. If you right now publish a .mjs file alongside it, it will be ignored until node has ESM support unflagged (or when the flag is enabled), at which point it will silently start working.

[pakastin]

How is module more "nonstandard" than .mjs? I think it would be way better idea in node.js as well.

If you would want to import module-"bundle", you use import, otherwise you use require. import would not support CJS and require would not support ES2015. That would be the best solution IMO.

[ljharb]

@pakastin "module" is something that two tools in the ecosystem decided to use. .mjs is something the platform itself is standardizing on. https://github.com/nodejs/node-eps/issues/57#issuecomment-336258618 is why "module" wouldn't be appropriate for node. Separately, forcing import vs require means that consumers have to know the module format of your package - that's an implementation detail, and users should not have to know that.

[pakastin]

In the front end we need to know if a module is UMD, CJS, ES2015, ... That’s not nothing new and what comes to node.js that will get better in the future once the packages get updated. But the .mjs would stay, which is why I (and many others) think it’s worse choice.

[ljharb]

Yes, and the fact that legacy scripts, UMD, AMD, and node-style CJS all used .js was a huge mistake. .mjs avoids making it again.

[YurySolovyov]

I may be wrong but I think tooling does not really care about package modes at this point, webpack seems to eat both CJS and ESM for breakfast without noticeable issues.

[WebReflection]

You can publish in npm a transpiled module (transpiled into CJS, presumable), with the .js file extension, and it will never break due to ESM support.

Incorect. As explained in my blogpost (really dude, take 5 minutes and read it, you're mentioned too there), if you have this basic module:

export default function test() { console.log('ok'); };

and you transpile it via Babel and publish, if an ESM module will import that transpiled file as such, it will break:

import test from "module"; test();

This is because transpiled to CJS modules do not interoperate with NodeJS ESM loader. You need Babel to even import that module or you need to use test.default() instead of test() which in turns it means that the following sentence is also false

forcing import vs require means that consumers have to know the module format of your package - that's an implementation detail, and users should not have to know that.

The current status is that nobody can use modules published as CJS via ESM without knowing upfront if these modules published a default too.

This is non-sense land. What you advocate as solution for users lies practically in the real world and it makes adoption of full ESM solutions problematic.

If NodeJS would be able to import a transpiled CJS like an ESM then everything would be fine but that's not the case, do you understand the issue?

[vkurchatkin]

The current status is that nobody can use modules published as CJS via ESM without knowing upfront if these modules published a default too.

You can't use them from CJS without knowing that, too.

[WebReflection]

You can't use them from CJS without knowing that, too.

Exactly, so telling the story that .mjs was born to hide module implementation details to users sounds like FUD.

Worst of all worlds is that we code in ESM, we publish losing the intent as CJS, and we're unable to use the same module as meant via a third parts ESM.

How user friendly is this? The story should be that we tell everyone to never publish CJS and force legacy to use @std/esm to import them.

Now that would be a moving forward pattern, the current one is just broken.

Looking forward to hear that telling projects to use @std/esm is unrealistic ... it's the easiest way ever to migrate to ESM, and it doesn't have .mjs contrains + it's not even close to that 80% of the Web.

[yawetse]

I've stayed out of the conversation for awhile, but having users understand the basic implementation details of a module I don't think is too much to ask.

Also wouldn't it be easy to provide really verbose error messaging Module x cannot be used with require, please import x from './x' or Module y cannot be used with import, please require(y).

That way CJS/require becomes a Node specific standard library for legacy modules but moving forward everyone migrates to the ESM standard

I think it's a lot easier to say, always use require for CJS and import for ESM (I know this has been said a million times already in this thread and other threads)

[WebReflection]

I think it's a lot easier to say, always use require for CJS and import for ESM (I know this has been said a million times already in this thread and other threads)

Exactly, and @std/esm is the perfect migration tool for that. Configure it as following and you're done.

{cjs: true, esm: "js"}

Instead, NodeJS is bringing the mess transpilers and early adopters created themselves as core issue, which is somehow promoting in the future the usage of whatever destructive technique they want, the community will just invent some workaround for them ... right?

This looks like a community failure and we should learn from it instead of promoting publication of CJS and making modern, standard, JS, not inter-operable with itself across environments by default.

At that point, using .mjs or .py would be basically the same.

[yawetse]

@WebReflection the real question is, will this GH thread have any meaningful impact on the decisions of the node TSC.

I know @bmeck has spent a lot of time and effort responding to threads like this over the past year, so I wonder if the .mjs extension solution is final

[YurySolovyov]

I think given the experimental status of ESM in node, it might be not final yet, but very close to it.

[yawetse]

@YurySolovyov maybe the better question is, what would it take to change the decision?

If I recall correctly, about 6 months ago, Unambiguous JavaScript Grammar was the solution.

[YurySolovyov]

If I recall correctly, about 6 months ago, Unambiguous JavaScript Grammar was the solution.

Right, but that turned out to be problematic and not good enough. See the @bmeck's table and comments at the top of the thread.

For me it seems like an obsession with focus on the "Smooth transition" and not a good engineering decisions.

[bmeck]

That way CJS/require becomes a Node specific standard library for legacy modules but moving forward everyone migrates to the ESM standard

ESM syntax does not mean that the module you are importing is ESM. It also has things in the works like HTML Modules that also are environment specific just like CJS. The ESM standard is not about only being able to consume one kind of module. This also includes things like WASM Modules, Binary AST, and WebPackage. Most likely more types of modules will be introduced into the ecosystem. I am not swayed by any argument that only accounts for ESM importing ESM and no other kind of modules.

I know @bmeck has spent a lot of time and effort responding to threads like this over the past year, so I wonder if the .mjs extension solution is final

Technically it has been final since last November. Additional things can be added but that is mostly being left up to custom loaders. People even in this thread have differing designs on how they want to make modules work and I will ensure that loaders are able to do any reasonable task, but .mjs as the default/standard way is final and also agreed upon as the only file extension in the MIME registry with a usage restriction to be ESM per @ljharb 's concern. Please take any alternative ideas to loader use cases at this point.

@WebReflection

I have talked with you in private, but I understand your concerns with the state of the experimental support. PRs are still landing and the support is not feature complete and as I said only a few days ago, you will have to wait. You will need to wait on various things like babel or webpack producing compatible output. Right now using them produces modules that do not act like ESM and cannot be loaded as ESM. Various approaches like the one in this gist exist to show real interop with non-ESM and ESM in the browser for example that they can use to output ESM as a first class format. Named import concerns about the shape of the module differing if you load .js vs .mjs are solved via a pragma but we are getting push back from transpilers about it making the transition too smooth and they worry about people never migrating since that means people can always ship CJS+pragma. In particular excluding the extension allows intelligent path finding to find .mjs for import and .js for require. The web is still trying to figure out path aliasing and bare imports. I am in no rush for making moves without their input as those hooks will be necessary in the browser to remove a build step for the ecosystem. You can track this issue if you want, but they are waiting ~6 months on people trying things out in userland.

Using cjs: true is an unlockable that introduces invalid imports from non-local CJS (dependencies and things in Node.js Core). Be wary when using it because it won't have the same effects as a pragma which is statically analyzed.

I am going to close this issue with the assumption that we will be solving these with loader hooks that have landed in core. Those hooks are likely to change a lot before being unflagged but I am happy to hear about use cases that people may need.

@YurySolovyov

For me it seems like an obsession with focus on the "Smooth transition" and not a good engineering decisions.

Maybe, but I disagree.

[WebReflection]

I have talked with you in private

that was before I've proposed .m.js and filed a PR.

Did you omit on purpose commenting about .m.js alternative that works out of the box everywhere and requires 20 chars changes in NodeJS to work already?

What would .m.js not address, practically speaking, that .mjs does? If we talk about MIME registry the Web already has one for JavaScript and if you tell the story only .mjs is registered then we're back to the fact .js is not, yet it works everywhere else.

Like I've mentioned in the DM, I also wish I was involved before making any decision final but like it's been for many other standards in ECMAScript, and to cite Mr Eich on making classes methods not enumerable by default the week before ES2015 was released:

it's not done until it's done

We're in experimental phase, and apparently nothing will be official 'till Node 10

Can we keep an open mind about all the addressed outstanding real-world issues?

Nothing is smooth right now, and it's not looking good for the future of the language with a dual extension nobody wants when the specification talks about modules, not extensions.

The future of JS is a made up extension that is not backward compatible and is full of hidden gotchas in terms of portability.

I hope we can fix this mess before it's too late.

Thank you.

[WebReflection]

One last thing: there are still trade-offs about goint .mjs direction, closing the thread won't make them disappear.

[bmeck]

Did you omit on purpose commenting about .m.js alternative that works out of the box everywhere and requires 20 chars changes in NodeJS to work already?

I believe this extension was discussed a long time ago if you dug up the discussion about it. It even had metrics gathered for it.

Can we keep an open mind about all the addressed outstanding real-world issues?

I agree, we can solve the issues; but we seem to disagree on how to solve the issues. That is why I am telling people to help ensure the loader hooks are enough for their own personal solutions that seem to vary.

Nothing is smooth right now, and it's not looking good for the future of the language with a dual extension nobody wants when the specification talks about modules, not extensions.

Correct, both extensions and MIME types are outside of the ECMA262 spec. However, they are vitally important for environments implementing ECMA262.

The future of JS is a made up extension that is not backward compatible and is full of hidden gotchas in terms of portability.

.mjs was explicitly chosen because it does not introduce backwards compatibility. However, moving forwards will require people to change their code from a problematic situation to a more clean and clearly defined situation.

One last thing: there are still trade-offs about goint .mjs direction, closing the thread won't make them disappear.

Correct, there are trade offs in all solutions. I don't see this as changing no matter what solution is chosen.

[WebReflection]

moving forwards will require people to change their code from a problematic situation to a more clean

What is exactly not clean or clear in the following code ?

import utils from './lib/utils.m.js';
utils.log('this is perfectly clear to me');

Not only it's clean:

• it work on every JS environment (SpiderMonkey, JSC)
• it works on every up-to-standards browser too
• it requires zero changes in the entirety of the Web to serve ES2015 modules
• it requires zero changes in the entirety of the tooling / IDEs involved software
• it requires zero changes in every OS able to preview or relate already .js files
• it uses a clear convention for a module, same convention used by .mjs as chosen name: m stands for module
• if the loaded file is not ESM it fails like it would fail on browsers and on NodeJS too loading invalid .mjs

Here the list of cons:

---

Correct, there are trade offs in all solutions.

What is the trade off of .m.js ? Where is the list of pros for the entirety of the JavaScript community and not just ad-hoc solution for the only env that has issues with it: NodeJS ?

[bmeck]

What is the trade off of .m.js ? Where is the list of pros for the *entirety of the JavaScript community and not just ad-hoc solution for the only env that has issues with it, NodeJS ?

Lets start with all language standard libraries agreeing that file extensions are based upon the characters after the last . present in the file path. This would be an oddity that causes problems with any tooling seeking to be intelligent about what mode a file is in. In addition, almost all MIME libraries have problems with this since they work on the agreement that the file extension is after the last . in the file path. That means they continue to be unable to determine the proper goal parameter and parse goal of a .m.js file since they only process .js.

This also has problems for things like require.extensions which only uses the file extension, which is to say that it only processes the characters at and after the last . in the file path.

Operating systems are the same. File associations are done based upon file extension.

Exceptions also exist of course. https://cgit.freedesktop.org/xdg/shared-mime-info/ for example does glob based file path matching instead of purely on the file extension.

The downside remains that tools processing .js cannot know the nature of a file with .m.js because they quite largely agree to work on file extension rather than complete file path. Under the circumstances of using the extension of .js which is what .m.js I see it as more of a problem than trying to do alternatives. It would change the entire notion of what file extensions are for systems.

The downside also encourages using an already tainted extension as has been brought up a few times in this thread. .js has no clear meaning except that it is somehow usually related to JavaScript. UMD/AMD/CJS/Flow/JSX/etc. continue to pollute this file extension and the idea of reusing .js only encourages a more polluted problem of determining what a .js file is.

Node.js chose to use .js and also carried fault, but it is going through the process of clarifying what .js means by adding a MIME registration in preparation for loader hooks.

it work on every JS environment (SpiderMonkey, JSC)

However, these environments do not work with various behaviors that the ecosystem relies upon and do not have a clear story for what even loading .wasm means and seem to load it as ESM. They need to look at and have a solution to alternative module types before I can see them as a goal of interop.

it works on every up-to-standards browser too

So does .mjs when served with the proper MIME.

it requires zero changes in the entirety of the Web to serve ES2015 modules

Most of the web is using MIME libaries that also need to find upgrade paths for WASM/Webpackage/etc. You will notice a trend here that they need to be able to upgrade MIMEs anyway.

it requires zero changes in the entirety of the tooling / IDEs involved software

This one is not entirely true. You are conflating tooling that needs/wants to know what kind of JS a file is and those that are fuzzy syntax highlighting like IDEs.

it requires zero changes in every OS able to preview or relate already .js files

.js loads JScript through WScriptHost in windows which I'm not sure is a compelling argument to see it as a good choice for ESM since it won't be supporting ESM. Same as WASM, etc. it just needs to update the MIME/file extension associations. Which would be problematic with .m.js since they are using the characters after the last . in the file path.

[WebReflection]

A file with a .js extension is a JavaScript file with the goal of being a file that respect ECMAScript specifications and it should work as such like it is already for SpiderMonkey, JSC, and every up-to-standards browser.

The MIME should be consistent and on the Web, where JavaScript comes from if you remember, the MIME for .js is valid and allows browsers to import files with such extension.

It would change the entire notion of what file extensions are for systems.

No. The file is a valid JavaScript file, like it's always been since the origin of its usage.

NodeJS arrived after JS, not vice-versa. NodeJS should understand what is JS, not vice-versa.

Node.js chose to use .js and also carried fault, but it is going through the process of clarifying what .js means by adding a MIME registration in preparation for loader hooks.

NodeJS is diverging on purpose condemning the language that made it famous.

The language follows specifications that talk about importing and exporting modules, it doesn't tell anything about extensions to define modules.

.js is a universal identifier for JavaScript files on the Web, where it was born, and where it's most used.

do not have a clear story for what even loading .wasm means

Nobody cares about .wasm here, let's not diverge even on the topic itself. We are talking about valid ECMAScript 2015 modules as defined by standard, something that already works everywhere but not in NodeJS: SpiderMonkey, JSC (without flags), browsers.

Most of the web is using MIME libaries that also need to find upgrade paths for ? WASM/Webpackage/etc. You will notice a trend here that they need to be able to upgrade MIMEs anyway.

Did I mention this is not about wasm and has nothing to do with it?

That's a different topic/issue/problem that is meaningless today for this discussion.

Like I've said, if .mjs solves for you, then it would do the exact same (but better) .m.js.

The PR is one liner and doesn't block, change, complicate anything about .wasm.

You are conflating tooling that needs/wants to know what kind of JS a file is and those that are fuzzy syntax highlighting like IDEs.

Everything already works today via .js extension so ... I'm not sure I understand what are you talking about.

Which tools doesn't highlight your .js import/export these days?

I think you're stretching arguments a bit too much.

If these keep being unrealistic, or far away from what's the real-world situation, we're stuck in stubborn-land and I'll stop commenting since the curse is decided and there's nothing we can do, unless somebody else with common-sense from NodeJS core would chime in and help.

'cause if tools need to improve their highlighting, and it's also very funny you mention this issue 'cause GitHub still doesn't highlight .mjs, those same tools need to update to handle .mjs regardless.

Every argument is valid, and less realistically to happen, for .mjs too. Nothing will be better or faster via .mjs and .mjs doesn't really solve anything for tools.

Tooling solve for tools, which is what we already do without the need to compromise the rest of the world.

Tooling can do a filename.slice(-5) === ".m.js" too, which is all I'm asking to implement while it's experimental and let the community decide because so far, beside 2 or 3 people here, every developer I've talked to is furious about this .mjs unnecessary diverging extension.

.js loads JScript through WScriptHost in windows which I'm not sure is a compelling argument

No, it's not. You are registering the MIME for .js which has since ever exactly the same issue, right?

So you already deal with .js daily, let's go back to reality.

Same as WASM

Here the thing: there's nothing to do, compare, or talk about if we keep .js the same way you're moving forward regardless with or without .mjs

Not a single problem you've mentioned is a real-world issue or, if it is, it's exact same issue going on with .mjs except it requires updates, migrations, and adoptions from every environment.

I have stats and real .m.js files that work already and my tools are fine so is the entirety of the world serving .js ... they don't serve JScript executable for Windows, they serve JavaScript.

But I'm honestly losing faith in programmers ability of being pragmatic. There's nothing bad in simple solutions that works, and every new pattern is a new opportunity.

A change in extensions for the very same language to break interoperability when modules are already the present and surely the future is a huge NO that I've no even idea how it happened.

Best Regards

[YurySolovyov]

.js has no clear meaning

It's meaning is perfectly clear - it is file containing code in JavaScript programming language.

WASM

WASM is not JavaScript. As simple as that.

tools processing .js cannot know the nature of a file

ATM tooling seems to handle both require and import perfectly fine or at least good enough.

[bmeck]

It's meaning is perfectly clear - it is file containing code in JavaScript programming language.

.js files on npm clearly are CJS which is not part of ECMA262. In fact it doesn't even have a top level parse goal that matches. Hence the MIME being added to disambiguate. Similarly people are using things like JSX in their .js files that are also not part of ECMA262.

WASM is not JavaScript. As simple as that.

WASM is a module format that will be loadable using import. It cannot be ignored just because it is not from ECMA262. The same goes for HTML modules.

ATM tooling seems to handle both require and import perfectly fine or at least good enough.

"good enough" does not mean that the match the standard. They in fact haven't taken a lot of time to be more standards compliant, see the babel PR.

[WebReflection]

about letting the community decide, my last post for today (or at least for a while).

the community put itself into this mess which is clear today to mostly everyone.

listen to them (see consensus) while it's experimental, or drop the flag already because there's nothing to experiment anymore.

[bmeck]

@WebReflection lack of import.meta, import(), fully defined loader hooks etc. mean there is plenty to keep behind a flag. You seem to be rushing ahead without trying to plan for the future.

[WebReflection]

my future is today and it works already.

It's NodeJS lacking behind and rushing a non solution, creating problems for the future of the language, creating diverging meaning for two extensions that should simply follow the ECMAScript Standard.

.js and .mjs do JavaScript: that's a redundant situation nobody likes for any other file format.

Follow the standard, don't diverge from it. NodeJS is sadly choosing the latter one.

[WebReflection]

Similarly people are using things like JSX in their .js files that are also not part of ECMA262.

that's their fault ... or are you planning to create .nojsx extension too for the future?

[YurySolovyov]

It cannot be ignored just because it is not from ECMA262

"good enough" does not mean that the match the standard

So, what's more important? I'd say if standard is in conflict with Reality, you change the standard

WASM is a module format that will be loadable using import

But for WASM it is perfectly reasonable to require different ext.

[bmeck]

So, what's more important? I'd say if standard is in conflict with Reality, you change the standard

This was actually attempted a few times with "use module" and unambiguous grammar. They never were able to progress through TC39. The standard isn't going to change.

But for WASM it is perfectly reasonable to require different ext.

And if JS gains another parse goal? BinaryAST is already on the table. If the argument is that it is not JS that isn't true; if the argument is that it is a different grammar so is Module and Script; if the argument is that is binary there is no reason there mandate that ECMA262 won't introduce another text based grammar goal.

[yawetse]

I've been trying to look for the answer to my previous comment in the old threads, but can someone succinctly explain why https://github.com/nodejs/node-eps/issues/57#issuecomment-336439298 wouldn't work (technically)

It just seems like it would be easier to make the loading of modules explicit by the user instead of trying to parse what kind of module it is.

Again in practice, all developers have to know something about the module they are about to use (like if the semver is compatible with the old version they are using, or what the module does)

Asking them to understand if it's an ESM module and to use import, I just don't think is a very big ask. And for those who don't both to look, having a very clear error message seems like the simplest solution (from an end user perspective).

Unless there's a huge performance or technical roadblock with going down this route, I'd love to make a push for it!

[bmeck]

I've been trying to look for the answer to my previous comment in the old threads, but can someone succinctly explain why #57 (comment) wouldn't work (technically)

The problem is not on implementation it is on migration and creating an upgrade path. import() coming to the Script goal was vitally important to the migration path. The main goal in any migration is to get people to use the new feature/syntax. The arguments about mandating require/import for the different file types center on a couple of issues:

1. you must expose require to ESM. This means you are actively encouraging people to continue writing require which is not something the browser will ever implement. Tooling can provide that in the browser, but you are still encouraging the use of a non-standard function even though import works fine loading non-ESM based formats.
2. The transitive transition story is bad. If my module is CJS and I upgrade to ESM or move from ESM to CJS (that would be weird), all of my dependencies must change the source code they are using to load modules. The point of not knowing the module type of your dependencies is to get people to always write import. This point is non-effective if CJS cannot properly mock the shape of all ESM. There is a need to implement the pragma to allow this, but talking with bundlers/transpilers they are pushing back on it and trying to see if it can be avoided.
3. timing, the format of ESM loading would mandate that all ESM dependencies evaluate before any CJS dependencies. Swapping formats does not only change the way modules are imported, it also would change the timing.
4. this makes things that are stuck as CJS like Node core in an odd world where you must always use require. Effectively any module using core modules would be web incompatible. The suggestions that were brought up around this include a complete rewrite of Node's core, which is not realistic.

[YurySolovyov]

And if JS gains another parse goal? BinaryAST is already on the table.

That's up to BinaryAST proposal author to solve this problem. Why should we live in fear to break something that does not even exist?

there is no reason there mandate that ECMA262 won't introduce another text based grammar goal.

This is actually pretty speculative. ECMA262 will have to come up with compatible solution.

[bmeck]

TC39 common sentiment is "You're letting implementation drive behaviour - it's the wrong thing to do". I firmly believe in being conservative but prepared for the future because of this.

[WebReflection]

you must expose require to ESM. This means you are actively encouraging people to continue writing require which is not something the browser will ever implement. Tooling can provide that in the browser, but you are still encouraging the use of a non-standard function even though import works fine loading non-ESM based formats.

have you done any Web development in the last 5 years? It's a honest question because the rest of the world has bundled everything via Browserify in the meanwhile and keep bundling via others.

Today we still need to bundle for browsers for various reasons and it's a different matter but using require will signal that there are legacy modules involved, which is a clear migration path.

It's visible to developer eyes, not NodeJS parsers, what's outdated and what's new.

You don't need special tooling to understand that, and tools are helped understanding the developer intent.

Moreover, if this is landing you can import require from "module"; and that's it, you have your crystal clean migration pattern that scales with the present and with the future.

It also looks like it's way too easy to counter argument your own examples.

As example, people today are omitting the extension, which is already a non standard behavior that only NodeJS is handling on its way (cjs or esm or ... what else ...).

On top of that, people use daily NodeJS specific core modules that bundlers, such Browserify or others, will eventually bring in.

Are you anywhere/anyhow trying to stop developers doing the following?

import {promisify}  from 'util';

Isn't that a diversion from standards too?

The transitive transition story is bad. If my module is CJS and I upgrade to ESM or move from ESM to CJS (that would be weird), all of my dependencies must change the source code they are using to load modules. The point of not knowing the module type of your dependencies is to get people to always write import.

They cannot, as demonstrated already. If the module was an ESM transpiled as Babel, you need to import the module and then its .default.

The not knowing the module type of your dependencies argument is a lie regardless because NodeJS has automatic extension insertion that could change the nature of the imported module: epic fail.

timing, the format of ESM loading would mandate that all ESM dependencies evaluate before any CJS dependencies. Swapping formats does not only change the way modules are imported, it also would change the timing.

or to rephrase, it will simply respect standards. They can easily interoperate, nobody stops you from importing a c.js file, right?

import {all, cjs, deps} from './c.js';

where c.js ...

import {require} from 'module';
const all = require('all');
const cjs = require('cjs');
const deps = require('deps');
export {all, cjs, deps};

this makes things that are stuck as CJS like Node core in an odd world where you must always use require. Effectively any module using core modules would be web incompatible. The suggestions that were brought up around this include a complete rewrite of Node's core, which is not realistic.

The core can be resolved specifically a part and that's not an issue: if it can work today it can work tomorrow.

any module using core modules would be web incompatible

I really do believe you haven't done much Web development recently ... I think you'd have a completely different opinion on most of your arguments if you did.

[bmeck]

have you done any Web development in the last 5 years?

It's a honest question because the rest of the world has bundled everything via Browserify in the meanwhile and keep bundling via others.

You seem to be using ad hominem implications trying to discredit my knowledge increasingly. I would prefer you cease.

Moreover, if this is landing you can import require from "module"; and that's it, you have your crystal clean migration pattern that scales with the present and with the future.

I actively talk to @tbranyen on discord about things, yes. I have seen this.

As example, people today are omitting the extension, which is already a non standard behavior that only NodeJS is handling on its way (cjs or esm or ... what else ...).

Clarify how it is non-standard. ECMA262 does not have any instruction on resolving specifiers except and abstract hook defering to the host (Node/Browsers/Shells).

On top of that, people use daily NodeJS specific core modules that bundlers, such Browserify or others, will eventually bring in.

Are you anywhere/anyhow trying to stop developers doing the following?

No, nor are browsers concerned with creating modules that will be browser specific to my knowledge. This is increasingly clear with HTML modules which require pieces of the browser that do not make sense in Node.

My aim is to get them to use a syntax that allows people to write ESM as an polyfill/implementation of util in your example. That would allow people to use it in the browser without making people use a bundler.

Isn't that a diversion from standards too?

How / what standard. ESM can load non-ESM, see repeated mentions of HTML Modules, WASM, etc.

or to rephrase, it will simply respect standards. They can easily interoperate, nobody stops you from importing a c.js file, right?

I still need to understand what standards, neither https://html.spec.whatwg.org/multipage/webappapis.html#resolve-a-module-specifier nor https://tc39.github.io/ecma262/#sec-hostresolveimportedmodule seem to be related.

I really do believe you haven't done much Web development recently ... I think you'd have a completely different opinion on most of your arguments if you did.

If you repeatedly insinuate ad hominem I will be get moderation involved.

[jamesplease]

@WebReflection I love the perspective you're bringing to this thread, and I know this is a subject folks are passionate about, but I don't think anyone needs to be slingin' insults. @bmeck is being really awesome by engaging with us in this discussion, especially when .mjs has, to many people, already "won." Plus, he's super knowledgeable about all of this, and he is helping us point out possible issues folks might run into with .mjs alternatives.

A bunch of folks do web development every day who also believe that .mjs is the right solution. I don't think we can or should make the argument that the reason that they want .mjs is because they aren't web developers. I do think there are other good arguments we can make, though!

:v:

btw thank you both. This conversation on ES modules support has been great to follow 👌

[WebReflection]

Just to clarify, it's not my intention to insult anyone but I've read enough FUD on this matter that sentences like the following one makes me feel like somebody is fooling me:

Effectively any module using core modules would be web incompatible.

I am trying to bring data to the table, experience, real-world use cases and proofs, and I keep reading about WASM, or a Ruby bundler, or even the abandoned HTML Modules as argument to support an absurdly breaking change as .mjs is.

If people abused ES2015 specifications as CommonJS loaders, I don't understand why the entire ecosystem should pay for them.

I've never advocated for transpilers, I've never advoacte for early adoption, and gosh if I've tried to warn community and NodeJS about the mistakes we were going to do without keeping CommonJS in CommonJS.

So, my apologies to @bmeck if he felt attacked by my questions, but these were genuine.

I don't know NodeJS as much as he does and if I would state "node will never be able to use ES2018", 'cause maybe I don't know there are live transpilers, I'd like him to tell me that what I'm saying is incorrect far from current real-world node status.

Anyway, all I know is that NodeJS is crucial for the JS community and it's driving a file extension change that nobody I've talked to likes, not even TC39 members agree on this, and AFAIK mostly actually don't.

I'd like to have objectiveness, relevance, and an open mind here, not mentions to death standards or situations easy to solve or already solved for long time.

Thank you all.

[tbranyen]

@WebReflection I'm wondering if we should make separate pull requests:

• --esm and --cjs flags (default to cjs for as long as the community needs), --esm will be default eventually
• import.meta.nodejs.require - Provides the necessary backwards compatibility to support the 500k+ Node-CJS modules out there
• Support .mjs or .js for ES modules

I agree that there is no reason for Node to support loading CJS via import. Exceptions can be .node, .wasm, the Node standard core, etc. as they aren't directly related to CJS. The core must remain CJS to support loading CJS modules, and I'd expect to be able to access the core from ESM. Therefore Node must prime the module registry with those entries.

All of this sounds feasible to me, and sure there are trade-offs, but it's the right path forward in my mind.

[bmeck]

HTML Modules as argument to support an absurdly breaking change as .mjs is.

I think you are referring to HTML imports, not HTML modules? HTML modules are still alive and being specced.

not even TC39 members agree on this, and AFAIK mostly actually don't.

Correct that there is not agreement, but there is general support which you can dig around on social media and see. WHATWG is somewhat cheeky in including it in the examples of resolving a module specifier even though the standard does not deal with file extensions.