nodejs / node

Node.js JavaScript runtime ✨🐢🚀✨
https://nodejs.org
Other
106.52k stars 29.03k forks source link

When will CommonJS modules (require) be deprecated and removed? #33954

Closed trusktr closed 4 years ago

trusktr commented 4 years ago

Just curious if this is planned to happen, or is it something no one is thinking about yet?

mmarchini commented 4 years ago

@nodejs/modules has probably discussed this extensively, and so have many other project members.

Sharing my opinion on this: deprecation and removal of CommonJS would be extremely disruptive to the entire ecosystem, and would cause more pain to our users than Python users experienced with migrating to Python 3. As such, deprecation is unlikely to happen in the mid-term future, if at all.

ljharb commented 4 years ago

I don’t see any point in ever doing that - CJS isn’t legacy, it’s just one of two first-class module systems in node.

jkrems commented 4 years ago

I think it’s too early to come up with a clear timeline for this. We’re just approaching the first non-trivial module based apps being possible (without compiling from something else, at least rewriting imports). And there’s a huge amount of code in the ecosystem and even more so in various companies that’s based on CJS - much of which may be non-trivial to convert to ESM (since ESM lacks certain features and is fully async).

So - I wouldn’t go as far as Jordan and say “never” but I’d say “long enough that it could as well be never at this point”.

jasnell commented 4 years ago

Still way too early to even consider, especially given that ESM is still experimental

devsnek commented 4 years ago

Seems answered, closing

jsumners commented 3 years ago

I hate to revive a long closed issue, but I don't think this topic is resolved. At least one major module developer is planning to remove support for CJS from their modules by the end of 2021 -- https://twitter.com/sindresorhus/status/1349294527350149121

I think without a clear signal from core that CJS is deprecated, module authors are going to have a maintenance nightmare on their hands. As a result of the above announcement, @mcollina has asked the Fastify community if we should do the same https://github.com/fastify/fastify/issues/2847.

What we will have is a hodgepodge of incompatible modules and fragmentation throughout the community. CJS codebases that wish to remain CJS will have to try and find replacement modules, fork and maintain older versions of the modules they use, or refactor their code base to either load modules asynchronously or convert to ESM.

Personally, as a backend only developer, I have not seen any "oh I gotta have that" benefits to ESM that would convince me to shift everything over. But if core were to come out and say "look, keeping both around is going to be a problem so we are deprecating CJS," then I'd basically have no choice. And it would give clarity to the community at-large as to what their maintenance path forward should be.

cc: @sindresorhus for your own visibility to whatever discussion happens here.

ljharb commented 3 years ago

It isn’t deprecated, so no such signal should exist. CJS and ESM are both first-class module systems in node.

If a package switches to only ESM, that package has chosen to exclude CJS users, and those users will either switch to ESM, or more likely find alternatives that don’t capriciously exclude them (since providing an ESM wrapper on a CJS package is relatively trivial).

WebReflection commented 3 years ago

My 2c, today I wrote this file because CommonJS can still do things ESM cannot, and work in both cases (as that file does).

There was a lot of effort to make dual packaging reality, and for personal experience I can say it works fine with everything out there (bundlers, CJS only projects, ESM ony projects), plus cache invalidation/hot reloading is still extremely hard in ESM, so that choosing to ditch CJS when there are tools (ascjs/ucjs to name two) that makes dual packaging a no-brainer, beside the only constraint that is live bindings, sounds a bit "capricious" indeed.

mcollina commented 3 years ago

There is no plan to deprecate cjs.

jsumners commented 3 years ago

There is no plan to deprecate cjs.

I am aware. My statement is that some discussion should be had about if a plan to do so should be created. In my view, there is clearly a painful future in store if it isn't deprecated.

To be clear, I have no desire to use ESM. But if having both CJS and ESM supported means there will be a fragmented community, then only one should be supported.

ljharb commented 3 years ago

There would only be a fragmented community if authors decide to exclude one part of it by shipping a non-dual ESM package.

The solution to that fragmentation is community pressure on those authors, and/or abandoning those authors.

jsumners commented 3 years ago

The solution to that fragmentation is community pressure on those authors, and/or abandoning those authors.

Sadly, I see myself doing that last part.

ljharb commented 3 years ago

That is also an effective form of community pressure.

jasnell commented 3 years ago

I think, ultimately, we need to come up with a new definition of "Deprecated" in Node.js. There used to be a very toxic notion in Node.js that an API or subsystem could be "locked", and that was used to justify all kinds of bad behavior around rejecting changes, fixes, and necessary improvements. I'm very happy that mentality has been pushed out of the project but we do need a formal notion of some API or subsystem that is just... done, or obsolete, even if it's not going anywhere, and even if someone wants to keep maintaining it. We have some examples of this already in core. The url.parse() API for instance. Yes it's deprecated but there's no plan on actually removing it. New code should never use it but there's no immediate reason to pull it and break existing code. I think CommonJS either is, or will quickly, get to that same point.

Whatever our personal opinions are about whether CommonJS was good enough -- it's not the standard, it's not where the ecosystem is going, it's not where JavaScript is going, and we need to just accept that fact and clearly communicate the direction we think things are heading in.

So, no, I don't think we're anywhere close to Deprecating CommonJS in Node.js... at least not from the point of view of removing it outright, but I do think we can and should draw a line in the sand and say that ESM is the direction we're going from here on out.

jsumners commented 3 years ago

So, no, I don't think we're anywhere close to Deprecating CommonJS in Node.js... at least not from the point of view of removing it outright, but I do think we can and should draw a line in the sand and say that ESM is the direction we're going from here on out.

How would core communicate that, in your opinion?

jasnell commented 3 years ago

How would core communicate that, in your opinion?

Would you take, "At this point I have no idea yet" as an answer for now? ;-)

jsumners commented 3 years ago

How would core communicate that, in your opinion?

Would you take, "At this point I have no idea yet" as an answer for now? ;-)

image

ljharb commented 3 years ago

I think CommonJS either is, or will quickly, get to that same point.

"new code should never use it" is objectively a false claim for CJS; there are a great many reasons new code should use CJS for the foreseeable future. If things change, then we can make changes at that time.

ESM is the direction we're going from here on out.

I think this would be the wrong statement for node to make - now or any time soon - and that stance would do great harm to the ecosystem.

jsumners commented 3 years ago

and that stance would do great harm to the ecosystem.

The harm has already been done by making two module systems available.

benjamingr commented 3 years ago

How would core communicate that, in your opinion?

Possibly by releasing a major version where esm is the default for files and a --module=cjs flag to opt-out of esm rather than opt in (and asking npm/yarn to make esm the default in npm init).

robpalme commented 3 years ago

ESM is the direction we're going from here on out.

I think this would be the wrong statement for node to make - now or any time soon - and that stance would do great harm to the ecosystem.

Maybe we should also consider the consequence of Node being perceived as directionless on the matter. Browsers are clear. Deno is clear. Modern tooling is clear.

Import Maps just landed in Chrome Stable today. We have already leveraged them to get portable/universal ESM code running across both client & server environments. The Developer Experience is amazing. The future is quickly arriving.

I think Node has a very healthy future. It's easier to sell that vision to others when we have coherent message that table-stakes features like ESM are the way forwards, even if rough edges (feature gaps) still exist today. CJS support will likely never go away, but in my personal opinion, it would weaken the Node project for it to appear confused about which way the world is going.

DerekNonGeneric commented 3 years ago

/to @jasnell

I think, ultimately, we need to come up with a new definition of "Deprecated" in Node.js. There used to be a very toxic notion in Node.js that an API or subsystem could be "locked", and that was used to justify all kinds of bad behavior around rejecting changes, fixes, and necessary improvements. I'm very happy that mentality has been pushed out of the project but we do need a formal notion of some API or subsystem that is just... done, or obsolete, even if it's not going anywhere, and even if someone wants to keep maintaining it.

Perhaps the term “effectively frozen” would be a good fit here.

effectively frozen
no new changes in the API surface unless in response to a major oversight

Refs: https://twitter.com/ryzokuken/status/1359612226009616386

ljharb commented 3 years ago

@robpalme where have browsers messaged that no new code should be written as Scripts?

I'll add, that TC39 won't be adding any language features that are ESM-only (besides ESM-related ones, ofc) - in other words, TC39 isn't stating that no new code should be written as Scripts either.

jasnell commented 3 years ago

@ljharb -- I would push back just a bit to make it clear: I'm not saying that Node.js users shouldn't have the ability to keep writing Scripts rather than Modules... and I'm not saying that the CommonJS support should go away. What I'm arguing for is that, given the direction the ecosystem is going, and given the massive differences that exist in the models, if we continue to keep our feet firmly planted on both sides we're not going to do either well. For better or worse (and I'm in the camp that it is indeed worse) ESM is the direction the platform has chosen and we should be emphasizing that moving forward (while continuing to support the legacy approach also).

ljharb commented 3 years ago

if we continue to keep our feet firmly planted on both sides we're not going to do either well

i don't agree with this assertion. If the impact on CJS (which constitutes the vast majority of the code on npm, since it includes transpiler output) is not considered when improving ESM, we're making migration from CJS to ESM harder, which imo makes it harder to ever get to a world that's primarily ESM.

In other words, I think the only way we can do either well is to keep our feet firmly planted on both sides.

bmeck commented 3 years ago

Possibly by releasing a major version where esm is the default for files and a --module=cjs flag to opt-out of esm rather than opt in (and asking npm/yarn to make esm the default in npm init).

They can do so now fairly easily by setting type:"module" by default, however changing the default mode was brought up in https://github.com/nodejs/node/pull/32394 , but doing so breaks many things. I am firmly against changing the default after playing around with having done so already.

robpalme commented 3 years ago

@ljharb I think @jasnell is communicating the big picture well.

To your point on browser and scripts, I don't think that is analogous to the situation Node is in. Browsers are unambiguous about ESM being the only natively supported module format.

guybedford commented 3 years ago

Ideally npm, package managers and tooling can start to default to adding "type": "module" on init, and warning when it is missing.

I previously tried to sketch out how the default might be possible to switch in https://gist.github.com/guybedford/40094b771176b0dbcaf6ab2e570c1acd. It does require coordination with package managers I think. There might even be a way to do it otherwise if anyone has bright ideas around flags as well.

theoludwig commented 3 years ago

In my opinion, it is better to make a standards way to use Node.js and not let too many opinions about how to use Node.js. I don't think it is great to continue to support both ways to do the same thing, and I actually think that we should gradually move all npm packages to ESM modules so we have a unified way to import JavaScript.

Of course, it will take a long time until everything is migrated but once done, it will make things a lot easier for newcomers and even for current users because it works pretty much the same way as every runtime for JavaScript (Browsers and Deno).

Seems answered, closing

from https://github.com/nodejs/node/issues/33954#issuecomment-646413467 by @devsnek I think we should reopen this issue, to continue the discussion, as even if it was closed, the discussion still happened, and we should still consider making CommonJS as legacy code ("Stability: 3" according to https://nodejs.org/dist/latest-v16.x/docs/api/documentation.html#documentation_stability_index) before removing/deprecating completely.

Also before doing anything, we should surely migrate the codebase of Node.js to ESM, because it still uses require internally.

I'm curious what are the reasons to use CommonJS instead of ESM in the future as claimed by @ljharb in https://github.com/nodejs/node/issues/33954#issuecomment-790019911 :

"new code should never use it" is objectively a false claim for CJS; there are a great many reasons new code should use CJS for the foreseeable future. If things change, then we can make changes at that time.

Actually having both CommonJS and ESM is confusing for everyone and it would be better to have a unified way.

braebo commented 3 years ago

I would just like to add that the front-end / fullstack-framework ecosystem (Svelte/Vue/React) has been phasing out Webpack in favor of the Rollup+ESBuild-based bundler, Vite.

Both Rollup's creator (Rich Harris Svelte creator) and Vite's creator (Evan You VueJS creator) have decided not to support CJS in favor of ESM, quoting:

"we should take the opportunity to push users to prefer ESM compatible libraries and move the ecosystem forward."

ESM / ImportMeta and family is quickly becoming non-negotiable for packages that want to support modern browser tooling, and many library maintainers are waking up to a stream of require|module|process is not defined GitHub Issues telling them that their libraries don't work.

Anyways, there seems to be a lot of confusion out there, especially for newcomers.

atian25 commented 2 years ago

Personally, as a backend only developer, I have not seen any "oh I gotta have that" benefits to ESM that would convince me to shift everything over. But if core were to come out and say "look, keeping both around is going to be a problem so we are deprecating CJS," then I'd basically have no choice. And it would give clarity to the community at-large as to what their maintenance path forward should be.

if we continue to keep our feet firmly planted on both sides we're not going to do either well

+1 for this.

As the author of multiple packages, the neutrality of the Node.js core team led me to be very confused.

I'm not sure how I should continue to maintain the hundreds of packages I currently maintain.

Whether to make a strong migration to ESM just like sindresorhus.

or just KEEP MY FEET ON BOTH SIDES -- which would only result in me being forced to choose TypeScript to write and compile to both CJS and ESM, but it calls as migrating from CJS to TS, rather than migrating from CJS to ESM, it's still harder to ever get to a world that's primarily ESM, and as we know, TS is not the standard either.

The Node.js core team should give us a clearer direction.

mcollina commented 2 years ago

@atian25 feature-wise ESM is still lagging compared to what is possible today with ESM. You can see this as really popular tools such yarn, jest, datadog and others are slowing adding support for it. This is due to the loaders implementation not being stable yet.

Stabilizing loaders will likely require a few years of work (not my estimation), so until that is done and ESM has feature parity with CJS before we could consider marking commonjs legacy or deprecated.

Given the network effect of NPM, I do not expect we would be able to ever remove commonjs or even deprecate it, so we will have to deal with commonjs modules for the foreseeable future.

My advice (not the TSC) for fellow OSS maintainers in 2021 is:

  1. if your module target the Browserv only, go for ESM.
  2. if your module target Node.js only, go for CJS. Make sure that exports work by following the rules in https://nodejs.org/api/esm.html#esm_commonjs_namespaces.
  3. If your module is isomorphic, dual publish it.

This will likely result in the lowest disruption to your dependants and users.

GeoffreyBooth commented 2 years ago

Stabilizing loaders will likely require a few years of work (not my estimation)

Hi, I’m organizing the loaders work 👋 and yes, it will take a while. Hopefully closer to one year than many years, but there are only two or three people sporadically working on the loaders roadmap, so progress is slow. We have a lot of agreement of the directions we want to go in, however, so really the main thing holding us back is people with time to devote to helping build it. If that sounds like you, please head over to https://github.com/nodejs/loaders and take a look at the TODO list and open issues and join us!

Other than that, I second @mcollina’s recommendations, although I also speak only for myself. The list of loaders use cases is more or less a summary of where ESM is still lacking in comparison with CommonJS.

XadillaX commented 2 years ago

@atian25 feature-wise ESM is still lagging compared to what is possible today with ESM. You can see this as really popular tools such yarn, jest, datadog and others are slowing adding support for it. This is due to the loaders implementation not being stable yet.

Stabilizing loaders will likely require a few years of work (not my estimation), so until that is done and ESM has feature parity with CJS before we could consider marking commonjs legacy or deprecated.

Given the network effect of NPM, I do not expect we would be able to ever remove commonjs or even deprecate it, so we will have to deal with commonjs modules for the foreseeable future.

My advice (not the TSC) for fellow OSS maintainers in 2021 is:

  1. if your module target the Browserv only, go for ESM.
  2. if your module target Node.js only, go for CJS. Make sure that exports work by following the rules in nodejs.org/api/esm.html#esm_commonjs_namespaces.
  3. If your module is isomorphic, dual publish it.

This will likely result in the lowest disruption to your dependants and users.

I agree with @mcollina.

If a package or a project is Node.js only (Pure server-end), I prefer CJS.

And if it's running on front-end (or dual-end), I prefer CJS too.

But the whole community seems like to use ESM on front-end (or dual-end). I am neither for nor against this.

marcus-sa commented 2 years ago

My 2c, today I wrote this file because CommonJS can still do things ESM cannot, and work in both cases (as that file does).

There was a lot of effort to make dual packaging reality, and for personal experience I can say it works fine with everything out there (bundlers, CJS only projects, ESM ony projects), plus cache invalidation/hot reloading is still extremely hard in ESM, so that choosing to ditch CJS when there are tools (ascjs/ucjs to name two) that makes dual packaging a no-brainer, beside the only constraint that is live bindings, sounds a bit "capricious" indeed.

When and why would you ever do that? Sure, lots of things you CAN do, but that doesn't mean you SHOULD.

Maintaining multiple package resolutions is a nightmare. A unified one like ESM would be way better.

As much as I love JS, one of the worst things is that it's not strict and opinionated - hence why TS is better.

ljharb commented 2 years ago

@marcus-sa that can’t ever happen though - there’s too much investment in CJS. Things would be worse, not better, if we stuck our heads in the sand and tried to pretend CJS didn’t exist.

Debating TS vs JS is off topic here; I’d be happy to educate elsewhere :-)

WebReflection commented 2 years ago

@marcus-sa

When and why would you ever do that?

when a module is optional, and doesn't work on browsers, workers, doesn't work on some arch, and so on ... glad you never had an optional dependency that might not be desired because it's 100MB of code nothing can, or will, use, as canvas is, in that case.

Maintaining multiple package resolutions is a nightmare.

There are at least 5 tools that make dual packaging, from ESM, or CJS, a no-brainer, and all my packages are like that, so I have no idea where this "nightmare" comes from: dual packaging is a quick and simple build step, like the one you need to do with TS every single time, except you need to do it only before publishing, when it comes to dual packaging.

marcus-sa commented 2 years ago

@marcus-sa

When and why would you ever do that?

when a module is optional, and doesn't work on browsers, workers, doesn't work on some arch, and so on ... glad you never had an optional dependency that might not be desired because it's 100MB of code nothing can, or will, use, as canvas is, in that case.

Maintaining multiple package resolutions is a nightmare.

There are at least 5 tools that make dual packaging, from ESM, or CJS, a no-brainer, and all my packages are like that, so I have no idea where this "nightmare" comes from: dual packaging is a quick and simple build step, like the one you need to do with TS every single time, except you need to do it only before publishing, when it comes to dual packaging.

I was specifically referring to the example file he provided.

Yeah, that's why you have package.json resolution depending on your environment.

I suppose you've never had to actually build and publish multiple packages with support for every module resolution, because most maintainers either don't do it correctly or don't do it all.

WebReflection commented 2 years ago

@marcus-sa

I was specifically referring to the example file he provided.

by he ... you meant ... me ??? ... yes, I wrote that, and presented the use case.

I suppose you've never had to actually build and publish multiple packages with support for every module resolution

Please read again: that is what I do already! I wrote posts about how to do that, I have tools to do that, and nothing is a nightmare.

P.S. I have no interest in this conversation, as it's bringing no objective points, nor arguments, to the plate, so I won't likely reply further around the discussed use case, as it's there because it was needed already, hence something to consider, thanks.

rilysh commented 2 years ago

This is an old issue, but I saw many conclusions where it's now a topic. Honestly, I don't really want to see Node going to remove CJS support in the future. I feel like CJS right now is being treated as a fallback rather than deprecated in the community. However, certain great node packages have already dropped support and moved to ESM. I see the point of doing this, but most of the node packages were written in CJS, and built-in node stuff as well. There's no point in doing this, they both work great, ESM just has more features but it doesn't mean to be we should drop old CommonJS already.

ghost commented 1 year ago

I see some other issues here. How will UMD modules work with import [from]/export [default]? That will involve a lot of decisions to be made and mass-migrations in the entire JS ecosystem - including other transpilers, such as PureScript, ReasonML, Nim, and Elm; we may need to replace AMD, UMD, CJS, and browser globals with ESM to make the migration simpler. However, we cannot be left behind the standards; if we want to add more features (ex. decorators and JSX) to ECMAScript, we will need to eventually migrate from the other module systems to ESM.

I think we need a larger discussion as soon as possible, so we can get the headaches of migrating out of the way. I wonder if @github-staff can start a discussion about standardization and the legitimacy of ECMAScript.

jsumners commented 1 year ago

I see some other issues here. How will UMD modules work with import [from]/export [default]? That will involve a lot of decisions to be made and mass-migrations in the entire JS ecosystem - including other transpilers, such as PureScript, ReasonML, Nim, and Elm; we may need to replace AMD, UMD, CJS, and browser globals with ESM to make the migration simpler. However, we cannot be left behind the standards; if we want to add more features (ex. decorators and JSX) to ECMAScript, we will need to eventually migrate from the other module systems to ESM.

None of that has anything to do with CJS and ESM: the two module types that Node.js supports.

I wonder if @github-staff can start a discussion about standardization and the legitimacy of ECMAScript.

What?

ljharb commented 1 year ago

Additionally, UMD already works fine - it’s treated as CJS by anything that understands modern module systems.

ghost commented 1 year ago

Never mind. :man_facepalming:

Haringat commented 1 year ago

There is no plan to deprecate cjs.

CJS will probably never be removed from Node.JS as there are thousands of unmaintained packages out there in the NPM ecosystem that use it and even more packages relying on those (for better or worse).

However, CJS will lead a shadowy existence in the future as it provides no real advantage, but a number of disadvantages when compared to MJS (e.g. no top-level await, cannot use MJS libraries) so fewer and fewer people will use it for new projects. At that point it will probably become deprecated, if only because people will have problems using it due to no interoperability with MJS libs and it will be like domains. Deprecated, but probably forever in Node.JS as a relic of times past.

ljharb commented 1 year ago

@haringat "no real advantage" may be true in the future, and i hope it is - but it's not true now, since loaders aren't shipped yet, hot reloading still can't work without a memory leak, etc. However, time will tell if the ecosystem's long term response to ESM-only packages is "switch to ESM" or "use/create CJS-friendly packages".

Haringat commented 1 year ago

@ljharb "create CJS-friendly packages" is a headache, as you either simply wrap a CJS module with ESM (at which point you basically maintain a CJS package as the entire functionally relevent code is in CJS) or you have two instances of your module hanging around which is bound to cause state-problems. And yes, the problems you described are there, but I would call those teething problems as I believe they will (hopefully soon) be solved.

benjamingr commented 1 year ago

No one is stopping you from exporting your esm code to cjs :]

ljharb commented 1 year ago

@Haringat or you just make a CJS-only package and you're done.

@benjamingr you can't do that synchronously.

Haringat commented 1 year ago

@ljharb or I just make a ESM-only package and I'm done.