what's the performance overhead like?

[capaj]

People can use ES6 modules for quite some time with transpilation(babel). How is this different from running a babel with only a transform-es2015-modules-commonjs plugin? Is it okay to use in production? How much does it impact the startup time/memory? Because babel eats memory like crazy even with that single transform.

[jdalton]

Hi @capaj!

The core of @std/esm, reify, has been in development for more than a year and is even in production use at Meteor. The ESM loader is a bit different than Babel. The @std/esm loader is intended for runtime use, so no build step or pre-publish step is required of the user. When it is released it will be fine for production use. In fact Lodash, the most directly depended upon package in npm, will itself depend upon @std/esm. The esm loader is intended to be quick and impact loading as minimally as possible. There is a 1 time transform cost per ESM module, after that the result is cached and loading is on par with CJS. I'm working to reduce the 1 time transform cost too. We can do this by avoiding a full parse and instead performing only a top-level parse in many situations.

The loader itself when published will be a small, ~40kb, zero dependency package:

[andywer]

Might be worth adding to the README 🙂

[akowald]

I noticed around 2x memory usage and startup time in my express/ws app with the default settings. Since I only esm for a single import this is hard to swallow. Is there anything you can recommend trying or is this just par for the course?

[jdalton]

Hi @akowald!

Over the last year performance has slipped a bit. I'm in the process of reducing startup back to microseconds from native.

As for memory that could be because we use V8 code cache APIs to improve startup and offset some costs (so probably a memory/perf tradeoff). The V8 code cache is for all code the loader touches (CJS and ESM) so if your entire app is run through that one import then it will code cache the entire app. We currently don't have an option to disable V8 code-cache beyond disabling caching all up (though a more granular config is possible). That said, I haven't tracked memory usage so a nice benchmark would be great (as I could dig in and get answers).

[akowald]

That's encouraging to hear! Of those two, startup time is probably more important.

In my project that one import doesn't have any dependencies. I tried putting require = require('esm')(module) at the top of the file that imports it but I get this SyntaxError: Unexpected token import. So I was forced to go back to -r esm or putting that line at the top of the entry/index js file were it works just fine. I'm probably missing something.

As for a benchmark, anything you come up will be better than what I tried. The following demonstrates the behavior I ran across. I created a simple project with express-generator: express --view=pug test. Then I modified www to spit out some basic information:

~/projects/test$ DEBUG=test:* node ./bin/www
  test:server Time to end of www 143 +0ms
  test:server Listening on port 3000 +3ms
  test:server Time to onListening() 146 +0ms
  test:server rss 35.86 MB +1ms
  test:server heapTotal 17.33 MB +0ms
  test:server heapUsed 8.49 MB +0ms
  test:server external 0.07 MB +0ms
~/projects/test$ DEBUG=test:* node -r esm ./bin/www
  test:server Time to end of www 430 +0ms
  test:server Listening on port 3000 +4ms
  test:server Time to onListening() 434 +0ms
  test:server rss 48.5 MB +0ms
  test:server heapTotal 31.33 MB +0ms
  test:server heapUsed 18.19 MB +0ms
  test:server external 0.29 MB +1ms

Hope that helps.. Oh I forgot to say thanks for esm, it works like magic! Edit: Should mention I'm on node v8.11.3.

[dnalborczyk]

I tried putting require = require('esm')(module) at the top of the file that imports it but I get this SyntaxError: Unexpected token import

@akowald do you have a repro for this?

[jdalton]

I'm guessing the created require call is in the same file as the ES import syntax (which is incorrect usage). The created require function should be used to load the ESM file.

[akowald]

You are correct, was just about to report back. It works now but at little benefit just due to other dependencies in the project. I could start making proxy dependencies but why bother at that point. Thanks again for the help.

[ken-okabe]

Hi @jdalton , this is out of topic, but since I don't use Twitter, excuse me to ask here. I'm a guy in electron/node issue, and you let me know this brilliant project. I would like to introduce esm concept/background in my blog in Japanese. Is it ok to translate https://medium.com/web-on-the-edge/es-modules-in-node-today-32cff914e4b https://medium.com/web-on-the-edge/tomorrows-es-modules-today-c53d29ac448c Thanks!

[jdalton]

@stken2050 That would be great! The newest (most up-to-date) post is the second one. During translation feel free to ping me with questions or to clarify anything. Let me know when it's published and I'll link to it from our docs too.

[ken-okabe]

@jdalton That is very kind of you, and yes, I will let you know things. thanks!! :)

[ken-okabe]

@jdalton I appreciate it again, and now I have a couple of questions regarding your article: https://medium.com/web-on-the-edge/tomorrows-es-modules-today-c53d29ac448c

Firstly, I was pleasantly surprised to know

In addition, package authors can now start new esm enabled packages using one of the following commands:

npm init esm (npm@6)
yarn create esm

https://docs.npmjs.com/cli/init

Create a new esm-compatible package using create-esm:

and the maintainer of create-esm is you.

So, I created the esm-compatible package, and found the package.json as

{
  "name": "my-esm-lib",
  "version": "1.0.0",
  "main": "index.js",
  "module": "main.js",
  "license": "MIT",
  "dependencies": {
    "esm": "^3.0.82"
  }
}

a new Node Modules Working Group has been formed (I’m a member), and the Node roadmap that once was isn’t necessarily what will be.

Defaulting to “Nodes rules” now, when those rules haven’t yet been ironed out, doesn’t seem like the right move. Developers should be able to pick up ESM and get coding fast. Ideally, ESM support should fall in line with where the ecosystem is today. With zero configuration esm should be as CommonJS (CJS) interoperable as possible.

However, because Node’s .mjs story isn’t fully written, the JavaScript ecosystem, as seen with Babel 7 and Webpack 4, can only responsibly support bare minimum functionality for it. Following their lead, esm locks down .mjs disabling all esm options. This means that today, .js is your best bet for things to just work. esm is a bridge from the ESM of today to the ESM of tomorrow. As Node’s path to ESM becomes clear, esm will allow developers to step into it.

So, here are my questions if I may ask:

1. Reading and considering the situation described above, I have a feeling with a hope you currently try to mirge esm to node and NPM, am I wrong?

2. Relating to the question-1, The article was written about 6 months ago, and I wonder what the most updated situation is.

3 >I’m teaming up with npm to enable npm init ! / npm init will soon be able to initialize more than package.json
I couldn't get the idea looking a the gif demo, is this related to npm init esm?

Thanks!

[jdalton]

@stken2050

Reading and considering the situation described above, I have a feeling with a hope you currently try to merge esm to node and NPM, am I wrong?

I don' think that's likely. However, you can manually compile esm into Node.

Relating to the question-1, The article was written about 6 months ago, and I wonder what the most updated situation is.

There hasn't been a lot of tangible progress on the official Node front. That said, I'm continuing to improve esm in areas like specification compliance, migration paths, performance, and real world usage scenarios.

I couldn't get the idea looking a the gif demo, is this related to npm init esm?

Oh, that demo gif was showcasing the npm init <create-package> feature. See this post for more info.

[ken-okabe]

@jdalton I understood:) Thank you for the prompt response!

[ken-okabe]

@jdalton It took more time than I'd expected, but finally I've published the translation of your article: Tomorrow’s ES Modules Today!.

---

Translation to Japanese

http://kenokabetech.blogspot.com/2018/09/es-modulesesm-jdalton.html

---

I did my best not to lose the context of your philosophy.

Although I tried to follow the original format as much as possible, the Twitter quote embedded script does not work well in the above link, however, for some reason, when the page is opened from the top page, http://kenokabetech.blogspot.com , the embedded script works. I tried to debug and fix it, but it appears to be due to Blogger issue.

Thank you for the brilliant article, permission to translate and your cooperation. If you find anything, please give me some feedbacks.

Thanks!

[EthanDM]

Is anyone else seeing significantly higher memory usage? I've got a medium sized Express app and it went from 100 MB to 250 MB.

[jdalton]

Hi @EthanDM!

There has been a mention above (with Express as well). While I have some ideas, I have nothing concretely actionable. If you have more details or insight (please post to a new issue) it would help me narrow things down and tweak things on our end or report a bug on V8's end.

[EthanDM]

@jdalton Just figured out what was causing a large part of the problem - it's related to the cluster module. Partly user error, but the issue wasn't as severe without ESM. I moved the cluster forking outside of my server JS file and it reduced my memory from 250 MB to 130 MB. It's still up from 70 MB, but that's something I'm ok with.

It looks like this:

if (cluster.isMaster) {
  // fork clusters
} else {
  require = require('esm')(module)
  module.exports = require('./server')
}