Implementation in another language?

[matthewmueller]

Hey guys, feel free to close this if it's out of place, but Node.js is pretty terrible language for heavily CPU-bound tasks. Even though this parser is heavily optimized, with large JS files it can get pretty slow.

Have you guys considered writing this in another language, like rust? I'm echoing a bit of the sentiment by @petehunt here: https://twitter.com/floydophone/status/764611095831072768

I'm not here to dismiss all the hard work that's been done here, just curious if this is something that has come up or if there's already work being done :-)

[hzoo]

The parser itself is going to be js for the foreseeable future as far as I can tell? I think we're busy enough just tackling issues and other things.

Other parsers can be found at http://astexplorer.net/ but they're all javascript except for flowtype.org or maybe https://developers.google.com/closure/compiler/ (java)

[loganfsmyth]

Performance is a tough. While a rewrite is always an option to optimize, at the moment Babylon is not the bottleneck in Babel's performance for the majority of usecases (mostly small files), so optimizing Babylon likely have a minimal effect on overall transpilation performance, and is unlikely to be a priority at the moment. For larger files, you may be right, I don't actually know, but large files are a relatively uncommon usecase for the average Babel workflow.

Node.js is pretty terrible language

Not to nitpick, but not a language :P

[hzoo]

I was going to say you can make your own parser and plug it into babel after https://babeljs.io/blog/2016/09/28/6.16.0#new-feature landed

but I see the tweet is about "golang version of webpack and speed up those builds" so it sounds like only the parser is relevant here and not actually about Babel itself?

[matthewmueller]

Not to nitpick, but not a language :P

Hehe, good call 😅

@hzoo yah maybe this is the wrong repo to discuss this. I just saw it as a starting point. But the goal would definitely be to replace webpack or browserify. It's crazy to me that even incremental builds take like 4-7 seconds on medium-sized projects. I can't imagine the swaths of engineers sitting around on big projects.

There's just so much frontend tooling built in JS, that it's a pretty long road to just make frontend builders outside of JS. Rust is potentially a good choice because Servo needs to build these parsers for their browser.

[hzoo]

Is it not worth just making webpack/browserify faster (or making acorn/babylon faster)?

[matthewmueller]

@hzoo Definitely not an expert in the internals, so I'm not sure how much more needs to be optimized, but it seems like they all claim that they're already really fast, which I think is true in the tests, but not so true in actual projects.

I wonder if it's worth optimizing further or acknowledging that it's a CPU bound task and should be done in a language better suited for those types of tasks.

[loganfsmyth]

My biggest concern is that there are no hard numbers backing this discussion. Reimplementing anything in another language is a ton of work, and without numbers, there's little to say that it wouldn't be better to just reimplement small parts of the codebase in other languages, or even just keep them in JS and optimize them.

One of my pet-peeves for instance is sourcemap processing, which is very CPU-bound. Maybe it could be reimplemented in another language, but maybe it could just be left in JS but rearchitected to avoid the unneeded processing entirely.

[matthewmueller]

@loganfsmyth yah, you're totally right. I was sort of probing to see if anyone else can already offer some more concrete information on this: saying this is a bad idea, not necessary, something we're already working on or a change we'd really love to see :-)

[danez]

I think what also needs to be considered before even switching is that building front end tooling in javascript, makes the audience of potential contributors to your project way bigger. If you are using some less common language like Rust, C++, OCaml, ... then a big portion of the people using the tool, might not be able to fix stuff on their own or contribute back, because they are not familiar with the language and how it works. I think this is also big part of the reason why webpack, browserify or babel are so popular because even though they are complex codebases, I might still find the reason for an issue that I experience or can make simple additions because I'm familiar with javascript and work with it on a daily basis and exactly know how it works.

A concrete example is that my company currently has major memory problems with node-sass/libsass, which is written in C or C++ and we cannot simple fork and fix, because none here is an expert in C and can fix memory leaks. So it makes us completely dependent on the maintainers (which in that case are very active luckily). sass/node-sass#1669