Providing a custom representation for an ES module under require(esm)

guybedford commented 3 months ago

What is the problem this feature will solve?

With the newly supported require(esm) we have the ability to require ES modules into CommonJS contexts.

One thing that might be useful is that since require() can return any JS type, while require(esm) will only ever return an ES module namespace, is to support customizations of the return value of require(esm) such that custom types can also be supported to retain the full expressivity of CommonJS modules in this interop.

Solving this problem can even allow CJS modules to upgrade to ESM as a non-breaking change, since any CJS module can then be represented by an ESM module under under such a rule.

Furthermore, such a pattern can also form the start of a new primitive for transpiling CJS into ESM, which may be the future of transpilers over an npm ecosystem increasingly migrating to ES modules.

When transpiling CJS into ESM it is critical that any CJS module can be properly represented in ESM when required by a real CJS module, which this would solve.

What is the feature you are proposing to solve the problem?

The feature is for an indication on the ES module itself to indicate to the CommonJS ESM import layer that the ES module has a custom representation to CommonJS.

For example:

export const __cjsDefaultMarker = true;
export default 'cjs module';

Where require(esm) of the above ES module would return the direct string 'cjs module'.

Further, I would like to suggest that we make this marker the same marker that is used to mark ESM CJS wrappers when importing CJS into ESM, per https://github.com/nodejs/node/pull/53848. The reason being that we then can ensure transitive interop.

That is, this marker supports both being created and being consumed in interop workflows. This is a requirement if this marker is to behave in a well-defined way in a CJS to ESM transpilation workflow as it is a requirement of interop patterns in that they can arbitrarily compose and "collapse" as they transitively lift and lower through the module system interpretations in various tooling workflows. A CJS module imported in ESM passed back into the CJS module system can then automatically be wrapped and unwrapped as required.

What alternatives have you considered?

No response

joyeecheung commented 3 months ago

Thanks for opening the issue, I like the idea of having a marker for require(esm) to unwrap default exports. I'd also like to see it applied to import cjs but I think to address the interop issue, the best way forward is to implement it the other way around from https://github.com/nodejs/node/pull/53848 - instead of adding the unwrapping marker to the synthetic module, what we should do is performing the unwrapping ourself when we see the marker.

Currently with what mentioned in the OP

export const __cjsDefaultMarker = true;
export default 'cjs module';

It gets transpiled to something like this:

module.exports.default = 'cjs module';
module.exports.__esModule = true;
module.exports.__cjsDefaultMarker = 'cjs module';

And imported by real ESM

import d from 'deps';  // d is module.exports, which is { default: 'cjs module',  __esModule: true, __cjsDefaultMarker: true }

This differs from the user expectation of "importing ESM from (transpiled) ESM" (as end users typically aren't fully aware of the transpilation going on):

import d from 'deps';  // Users expect d to behave as if being imported from authored ESM, so d should be 'cjs module'.

This was a oversight that has been bothering users ESM-to-CJS transpiled library users (e.g. see https://github.com/evanw/esbuild/issues/1719 or search for default export in bundlers/transpilers' issue trackers) . If we are inventing a marker that leads to the unwrapping of default exports, we should make it work for both require(esm) and import cjs otherwise we risk creating further disparity.

This also addresses the question in https://github.com/nodejs/node/pull/53848#issue-2407593101 when it comes to importing CJS -> ESM transpiled packages, if the transpiled package defines this marker, Node.js already does the unwrapping during import cjs phase, and the transpiled consumer gets { default: transpiledNS.default, ...namedExportsFromCjsModuleLexer, __esModule: true }, which it can then wrap with depMod.__esModule ? depMod : { default: depMod } easily.

joyeecheung commented 3 months ago

Solving this problem can even allow CJS modules to upgrade to ESM as a non-breaking change, since any CJS module can then be represented by an ESM module under under such a rule.

I don't think this is a thing we should advertise, upgrading from CJS to ESM have other breaking implications e.g. making the returned result immutable i.e. not mockable. I previously already received questions about why the result of require(esm) is not patchable from folks working on APM tools (and unfortunately this is in the ESM spec and is out of Node.js's control). The marker only serves to help CJS to ESM upgrade for library authors, but it won't be the key to make such upgrade non-breaking, it only helps making some libraries break less for end users if they have been replacing the module.exports objects with something special (for libraries that are only exporting an ordinary dictionary as module.exports and don't intend to have default exports after the upgrade, which is quite common, they don't need to use this marker at all).

nodejs / node