Please consider adding native support for ES modules

[kirlat]

Do you want to request a feature or report a bug? I want to request a feature. What is the current behavior? Right now Jest does not support test suites with import statement. They result in the following error:

SyntaxError: Unexpected token import

      at ScriptTransformer._transformAndBuildScript (node_modules/jest-runtime/build/script_transformer.js:305:17)
          at Generator.next (<anonymous>)
          at new Promise (<anonymous>)

What is the expected behavior? Would be great if Jest supported ES modules natively.

Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system. Jest: 21.2.1 node: 8.9.0 npm: 5.5.1

Before, native support of ES modules was not possible since node.js did not support them. Starting from a few versions ago, node.js added support of ES modules with a flag (https://nodejs.org/api/esm.html). It would be absolutely great if Jest would match this with adding support of ES modules too, probably with a flag or even without it.

Node.js requires ES modules to have an .mjs extension. In order to support ES modules Jest needs to add a recognition of those extensions. Jest will also need to pass an --experimental-modules flag to node.js until node will implement support of modules without a flag. I'm not sure if any other required changes would be needed within Jest in order to support this. I can only hope it will not be terribly hard.

Ideally, it would be cool if Jest would recognize modules even in files without .mjs extensions since code that targets browsers do not use them, but I don't know if it is ever possible. Node.js provides loader hooks for that (https://nodejs.org/api/esm.html) but this still doesn't solve the issue with a reliable determination of what type of module the file is.

I believe ES modules are a great feature vastly superior to all existing JS module solutions. Having it implemented in node.js opens a door for Jest to have it too. This would allow developers stick to using the first truly standardized JS module format not only throughout development, but trough testing as well.

[SimenB]

Jest has its own require implementation (https://github.com/facebook/jest/blob/master/packages/jest-runtime/src/index.js), so it would be way more involved than just supporting the syntax and default to looking at .mjs. I'm also against activating experimental flags.

It might be possible to automatically transpile import/export, but implementing the semantics is a huge undertaking, and probably blocked for a year because of support for node 6.

[cpojer]

I agree with SimenB. We need a number of hooks from the node team to be able to make this work together with the vm module. However, I think we should support this in the meantime, but not by mirroring the full native implementation but rather by using babel and compiling it to require inside of babel-jest. I think for testing purposes this will work fine and we do not have to provide the same guarantees the node runtime needs to provide anyway.

[SimenB]

So just adding babel-plugin-transform-es2015-modules-commonjs & babel-plugin-dynamic-import-node to babel-jest?

[cpojer]

Yeah, that's what I was thinking.

[TrySound]

Guys, what about integrating https://github.com/standard-things/esm ? It's fast and maintain a lot of edge cases.

[cpojer]

@TrySound what would that look like concretely? Can you make a prototype?

[SimenB]

We still have our own require-implementation (needed for mocks), so I don't think that'd help much.

And we need to work both with Node's rules and the browser's rules.

I'd be very happy to be corrected and have it work perfectly for us :D

[SimenB]

@std/esm apparently should just work with jest: https://twitter.com/jdalton/status/930257653292400640

Could anyone give it a whirl and come back with a PR for the docs? 🙂

[TrySound]

I guess users would like to have support everywhere, but I found it works only for dependencies of test files.

// test.js
require = require('@std/esm')(module, { esm: 'js', cjs: true });
const utils = require('./utils');

// utils.js
export { default as update } from './update';

It's better, but not ideal.

[uipoet]

https://github.com/tarunlalwani/jest-overriding-require-function-stackoverflow

[gaearon]

So just adding babel-plugin-transform-es2015-modules-commonjs & babel-plugin-dynamic-import-node to babel-jest?

I don't think this is a great solution, as it doesn't perform any "missing export" checks that are so valuable with ES modules. For example, in React repo I started running build more often just because Rollup finds these mistakes but Jest with es2015-modules-commonjs does not.

@std/esm apparently should just work with jest

It would be pretty great to invest some time into their interop. Pretty sure this hacky solution will break eventually: https://stackoverflow.com/questions/46433678/specify-code-to-run-before-any-jest-setup-happens. But if it's just a matter of exposing something on Jest side, would be cool to see it supported.

[awidjaja]

@SimenB, in my mind the immediate step would not be too complex. What is urgent is to allow people to work with .mjs module, even if babel is helping behind the test scene. Otherwise people might have to find different testing solution if they want to use .mjs.

1. Fix the testMatch property to allow .mjs file to be found (currently it doesn't work even the regex seems correct already, maybe there's a hard code somewhere rejecting .mjs extension)
2. Pass --experimental-modules flag when running node.js so that it will run natively
3. Jest should treat .mjs the exact same way as .js today (e.g. in jest require) -- the import statements are already allowed inside .js today, so only need to allow .mjs extension.

The final solution might be complex and takes time, but it is inevitable isn't it?

[jcguarinpenaranda]

Hello, has someone been able to fix this error?

[skkap]

We are using .mjs with "node --experimental-modules" option. Any workaround?

[jdalton]

We are using .mjs with "node --experimental-modules" option. Any workaround?

That's experimental and not fully fleshed out. There is lots of churn still with basic things, like how to import a builtin module, still up in the air. Projects like AVA have started allowing the use of @std/esm as their loader pipeline if used (bypassing Babel). Maybe jest could take a similar approach.

[SimenB]

Supporting @std/esm is something we want to do, help to implement it is more than welcome!

[jdalton]

@SimenB Can you chat sometime on hangouts?

[jdalton]

Hi @SimenB 👋

An esm user has contributed esm + Jest demo and I figured it might be a good starting point for creating a more official cow path.

Update:

The esm + Jest demo has been updated with basic module name mapping support.

[SimenB]

That's pretty cool! Thanks for sharing.

We'll have to figure out where we want the integration to be. How would it handle CSS (or other non-js assets) files? Should it just be a transform? What about the built-in babel transform? How should Jest behave when it comes to the incoming loaders, if it does affect anything?

[kenotron]

It seems like there may be a benefit for a community contrib esm-enabled alternate jest runner (or an unofficial / experimental flag) so we can make progress on something like that. Would there be interest in that from jest team?

[SimenB]

require is not implemented in a runner, it's in the runtime itself. Any contributions towards making it pluggable is very much welcome (ref #848).

I'm sure if you can get the example code @jdalton linked to to work without issues (or close to it), it should be simple enough to load in the esm loader behind a flag in jest itself. One thing I see as an issue is that it wants the real module global, not the fake one we create. Not sure if it means modules can leak between tests? I don't know what esm does with it under the hood. It also doesn't handle mocks, so mocks with import would still break

[jdalton]

May just need some tweaking of how Jest taps into CJS. For example when mocking the module object, instead of using a plain object, it could use require("module") and then wrap/overwrite module.require to intercept requests and juggle as needed.

Update:

I'm now working on enabling Jest out of the box with esm (with hopefully nothing Jest has to do differently). I'll continue to experiment over the weekend but early signs look good.

[JasonCust]

@jdalton Any update with the esm compatibility?

[jdalton]

Hi @JasonCust, wow my comment has gotten some attention!

I have made progress in identifying the work needed to enable Jest support in esm. In my experiments I got Jest loading and evaluating tests written in ESM. The work required on the esm loader side is to make the way we handle vm.Script use more generic. At the moment we hook into that primarily for REPL use which assumes a single module. We have to make that a bit more generic for Jest support to shake out. It doesn't look like Jest will have to change anything. The refactor of our vm.Script support is still on my TODO and will still be tackled before I release things like experimental WASM support. At the moment I've been squashing bugs that have popped up around improved APM and mocking support.

[JasonCust]

Thanks for the update @jdalton as I am excited to be able to use esm with Jest. So as to not bother you while you are working on those things, is there a task for esm that we can follow along with the progress?

[jdalton]

You can keep subscribed to this thread of follow the repo. Jest support will be in the the v3.1.0 release so you could keep a look out for that version.

[deepj]

@jdalton Any news about supporting Jest in esm?

[jdalton]

Hi @deepj!

It's still on my list and the items I can tackle before getting this done are shrinking. I've been improving supplemental Jest support of testing esm modules within Jest (though CJS within Jest tests). There's still nothing critically blocking implementation on Jest's side (so no work for them to do). Microsoft, my employer, is a heavy Jest user too so it's one of my day-job goals to improve this as well. I'm hoping to tackle it soon.

[deepj]

@jdalton Good to know. And thanks for your work. I appreciate it!

[manniL]

Really looking forward to this feature :speak_no_evil:

[yanickrochon]

Been trying to make this feature work for the past two hours. There are so many partial solutions, half-documented answers, all different from each other... It used to be easy to test node.js packages.

[Volune]

So just adding babel-plugin-transform-es2015-modules-commonjs & babel-plugin-dynamic-import-node to babel-jest?

What happened to that idea? Any issue in implementing it?

[suren-atoyan]

Hi @jdalton. I was wondering if you could keep us up to date about status of this issue. Sorry if I bother you, but I am waiting for this for a while, and it would be better if we'll receive an update at least for last/next six months. Thank you :)

[jdalton]

Hi @SurenAt93!

Thank you for your patience. I'm hoping to have a release out by the end of the month with Jest support. With that you'll specify esm as a Jest transform.

[kenotron]

Cool. How is that transform different than Babel?

[jdalton]

@kenotron

Using Jest's transform option is a way for me to side-load the esm loader. So while it is technically transforming source too, it's also wiring up the esm loader.

If the question is more what's the difference between Babel and esm. Babel is a collection of packages that transforms source, one of the targets could be ESM syntax. The esm loader is a zero dependency loader that simulates the runtime environment. So esm supports things like dynamic import(), passes relevant test262 specs (temporal dead zones, live bindings, upfront-errors, etc.), and supports loading a mix of CJS/ESM/WASM to taste by configuration.

[suren-atoyan]

@jdalton Thank you for you work and support!

[lukasholzer]

@tomheller ;)

[igasparetto]

Been trying to make this feature work for the past two hours. There are so many partial solutions, half-documented answers, all different from each other... It used to be easy to test node.js packages.

I agree.

I created a simple Vue project, which also manifests the problem. https://github.com/igasparetto/vue-jest-test I wasn't able to get it to work.

I followed the instructions from the following pages:

• https://vuejs.org/v2/guide/unit-testing.html
• https://medium.com/@kevin_peters/unit-testing-vue-js-components-with-jest-86e14ef499da
• Please see this thread: https://github.com/vuejs/vue-cli/issues/1584

My machine (not sure it should matter):

• Windows 10 Pro 64-bit
• Node v8.9.4
• Vue: 3.2.3
• npm: 6.5.0

[igasparetto]

@kenotron

Using Jest's transform option is a way for me to side-load the esm loader. So while it is technically transforming source too, it's also wiring up the esm loader.

If the question is more what's the difference between Babel and esm. Babel is a collection of packages that transforms source, one of the targets could be ESM syntax. The esm loader is a zero dependency loader that simulates the runtime environment. So esm supports things like dynamic import(), passes relevant test262 specs (temporal dead zones, live bindings, upfront-errors, etc.), and supports loading a mix of CJS/ESM/WASM to taste by configuration.

@kenotron could you provide us an update please?

[kenotron]

@igasparetto, it's @jdalton's work that should enable this. I hadn't tried that solution just yet.

[SimenB]

Latest status there I think is: https://twitter.com/jdalton/status/1080627279124934661

[jdalton]

Yep! Sorry it's taking longer than I wanted. Locally I now have all relevant test262 tests passing. In the course of getting those up the mocking related scenario tests slipped so I have to pick them back up before I can release. It isn't insurmountable, it just takes a bit of time. I'm presenting at Covalence Conf on Jan 16th and hope to have a release by then. In other news, npm tink early adopted esm for its ESM syntax support 🤝.

[igasparetto]

Jan 16th and hope to have a release by then

@jdalton I hope the presentation went well.

Have you got an update on this release please?

[manniL]

@igasparetto It's in https://github.com/standard-things/esm/issues/97#issuecomment-454985896 and in https://github.com/standard-things/esm/issues/706 ☺️

[mcarifio]

I'm leaving a small "data point" here to help you in your planning. I realize you are all donating your time and skills to address this issue. As a dev myself, I appreciate your contribution.

I just spent most of yesterday, a Saturday no less, to come up to speed on javascript modules on both the client and server side. ESM, CommonJS, AMD, what a confusing mess. I was never able to get a jest test to work with ESM loading a (node) module using a .mjs extension. I could load the same module successfully client side. I could create a node "client" that used the module with an import statement. I couldn't configure jest correctly to consume the same import statement, with or without babel, with or without esm. I eventually switched to ava, and just got it to work following a recipe on their website. Yes, I followed a recipe, I don't fully understand the machinery that made all the parts work. But at least I can now write ESM loading javascript modules and associated unit tests. I think. I'm extrapolating on the basis on one single success. They also have a recipe to connect ava to webstorm. But at least they are presenting recipes to mere mortals like me.

I also realize this message will read like I'm whining (which in part I am). I see all the work that's gone into jest. But this ESM support thing is a killer and a deal breaker for me. Thought you'd appreciate some feedback, but if not, ignore this or ask me to delete it and I will.

[dandv]

Any news in the light of the new modules support in the v12 release?

[azz]

@dandv I've been doing some investigation into if Jest can support native ES modules in node 12. It will involve Jest using the vm.SourceTextModule API, which requires a few CLI flags to be passed to node:

--experimental-modules --es-module-specifier-resolution=node --experimental-vm-modules

The API is super low-level too.

TBD.

Discussion in https://github.com/nodejs/node/issues/27387

[azz]

Update: been told to wait until the design of Node's module loader is locked down. In the mean time, standard-things/esm#706 might be the best bet.

[htor]

jest is a very nice testing library. support for esm really all it needs to be complete!

Update: been told to wait until the design of Node's module loader is locked down. In the mean time, standard-things/esm#706 might be the best bet.

this does not work with jest, sadly.