So... how is it SO FAST?

[kizerkizer]

[billywhizz]

:grinning: with great difficulty! there are obviously a lot of caveats and the code i wrote to get this performance was pretty ugly. i hope i explain it pretty well here.

having said that, i don't think there is anything stopping it staying around that level even with some nicer abstractions to work with (i have found some issues since that are actually causing it to perform worse than it should).

the main objective of getting it into techempower was to establish some kind of baseline for how fast it can go. so as features are added and complexity grows we can see impact on the baseline. when i started out i kinda hoped to get a decent score and didn't really imagine it would be nearly as fast as a heavily optimised c++ framework.

[billywhizz]

also, it is, deliberately, a very thin wrapper around linux system calls and libc. there are no c++ classes or complex object models or abstractions at all. just near raw access to the system interfaces. all the kudos has to go to the v8 team. the cost of calling from JS into C++ is very small now and if you are careful with not creating work for the garbage collector and don't have to do a lot of string manipulation the various levels of JIT do a marvellous job of producing highly optimised code.

[kizerkizer]

That's interesting because I hypothesized that a v8 runtime where most of the "heavy lifting" (e.g., HTTP request parsing, anything computationally heavy really) was done in C/C++ land would be faster than the minimal wrapper strategy (as I assumed crossing the boundary was slow). Goes to show how advanced of a JIT v8 has become. Makes sense when you have the world's most popular browser AND one of the most popular runtimes period (node) depending on the same engine; can't imagine how much work has been done on it by so many people.

Looks like the hypothetical case of a "super" JIT beating out native is a reality.

[kizerkizer]

I'll give your post a read. Great job!

[billywhizz]

That's interesting because I hypothesized that a v8 runtime where most of the "heavy lifting" (e.g., HTTP request parsing, anything computationally heavy really) was done in C/C++ land would be faster than the minimal wrapper strategy (as I assumed crossing the boundary was slow)

yes, this was the case in the past but no longer i think. i'll see if i can produce a little benchmark to measure the overhead. and the http parsing is done in c++ using picohttpparser which takes advantage of SSE for a slight edge. there's very little complex c++ in the codebase though. i was amazed how fast the postgres wire protocol in JS was.

[kizerkizer]

That's interesting because I hypothesized that a v8 runtime where most of the "heavy lifting" (e.g., HTTP request parsing, anything computationally heavy really) was done in C/C++ land would be faster than the minimal wrapper strategy (as I assumed crossing the boundary was slow)

yes, this was the case in the past but no longer i think. i'll see if i can produce a little benchmark to measure the overhead. and the http parsing is done in c++ using picohttpparser which takes advantage of SSE for a slight edge. there's very little complex c++ in the codebase though. i was amazed how fast the postgres wire protocol in JS was.

Ah, cool. Yeah, I assumed you had deferred to some tried-and-true libraries for things like that (which is perfectly fine of course). A benchmark like you mentioned would be informative.

It would be cool to see just how fast you can get this, though I guess your priority is stabilizing things/apis and getting docs written.

From what I understand, the "event loop" is implemented in js, since epoll is exposed directly to js, right? Also, for people wanting node compatibility, there's a project out there that implements all of the node apis in js and only needs a single syscall function to be passed in. I can't remember the name, but it would save you a bunch of time if you decide to support that (also, people could just use that project directly themselves). It's just handy since it opens up use of all of npm world.

Let me know if you'd like some help with anything in particular. This could find widespread use because linux in the cloud is almost a given now, and node is bogged down with all sorts of backwards compatibility/other constraints.

[billywhizz]

It would be cool to see just how fast you can get this, though I guess your priority is stabilizing things/apis and getting docs written.

yes! this is a priority right now. i have just been hacking away on my own for a while on this and doing various experiments so have not prioritised it but it needs to be done. there's still quite a bit of work to do to get a stable release in terms of organising the api's and standardising the way i do things across different modules.

[billywhizz]

@kizerkizer

From what I understand, the "event loop" is implemented in js, since epoll is exposed directly to js, right?

yes. the goal here was very much to expose as much of the linux syscall surface as i can to JS. i am not super-interested in being cross platform beyond good coverage on most mainstream architectures that run linux and maybe BSD support, but even that would require a lot of changes and abstractions.

i am not sure if exposing epoll directly is a good idea and might be better to abstract that away behind some simple api that would allow using epoll/kqueue and other event loops under the hood. what i do like about exposing it is it makes it very explicit to the dev that this is an event loop. there is nothing "asynchronous" going on here. you are synchronously polling a list of events in a non-blocking way and calling handlers as you receive them. this is the way all event loop systems work and i think it's important to expose that up front and not hide it behind some higher level api like streams.

one of the motivations of the project is also for devs using it to be able to easily understand what is going on and ideally to learn more about v8 internals, c/c++ and linux system api's in the process. it has a very small surface area at the moment. the core is only a couple thousand lines of code:

total files : 11
total code lines : 2099
total comment lines : 60
total blank lines : 128

just/just.cc, code is 440, comment is 16, blank is 25.
just/just.h, code is 97, comment is 0, blank is 12.
just/just.js, code is 364, comment is 8, blank is 29.
just/lib/build.js, code is 330, comment is 14, blank is 12.
just/lib/configure.js, code is 248, comment is 4, blank is 8.
just/lib/fs.js, code is 211, comment is 5, blank is 17.
just/lib/loop.js, code is 85, comment is 3, blank is 3.
just/lib/path.js, code is 110, comment is 1, blank is 9.
just/lib/process.js, code is 87, comment is 7, blank is 6.
just/lib/repl.js, code is 102, comment is 2, blank is 6.
just/main.cc, code is 25, comment is 0, blank is 1.

everything else is in the libs or modules repos. there is also a lot of boilerplate i could remove by using macros but i want to avoid macros and big abstractions for now as i think they get in the way of others understanding the code.

This could find widespread use because linux in the cloud is almost a given now, and node is bogged down with all sorts of backwards compatibility/other constraints.

yes. i have worked a lot with node.js and it has become increasingly hard to understand what is going on in the codebase, which make bug hunting etc. very difficult. imho it has grown far too large and suffers from not narrowing it's scope much earlier in it's lifetime. but you cannot argue with it's success and wide adoption. i don't really see just(js) competing with node.js as node.js is very much a general purpose platform. where just(js) might be useful is in the container/isolation space (i have been doing a lot of experiments with this) and on the server side if you need to get closer to the metal and do things that are just not possible or very difficult with node.js. does that make sense?

It's just handy since it opens up use of all of npm world.

yes. i can see the attraction in having access to all of npm but at same time supporting this would require adopting/supporting all the fundamental abstractions in node.js which i think have some pretty severe flaws (e.g. streams). i think it would be possible to do something like you suggest in userland and would fit in with what i see just(js) as - a small and low overhead substrate for building other things like node.js on top of.

Let me know if you'd like some help with anything in particular.

it sounds like you are interested in node.js/npm compatibility and right now i am not super interested in working on that but others have expressed an interest. maybe if you wanted to start a little side project and see what it would take to support something like express that would be useful. i am happy to help out with whatever takes your interest and try to get you up to speed in hacking on the codebase. it will be useful in helping to flesh out what docs/demos the project needs. i started a gitter here but i rarely check it. not sure what best options are for comms.

to summarise, i think i see this more as "c" in javascript as opposed to anything more abstract/OO like java/c++/c# etc. the heavier abstractions can live in userland.

[billywhizz]

also, and regarding npm/module systems and security, this by @guybedford is worth a read. i would be interested in a capabilities system like this for just and it really needs to be enforced at the module/package management level which makes building secure systems with node.js npm difficult as they are currently configured.

i have done some experiments with seccomp/namespaces/kvm etc. and all this stuff is quite easy with just as it is so close to the syscall interface and easy to add required functionality.

there is also an ffi binding that works quite well now and makes it possible to hook into c/c++ libraries just using JS.

[Arcitec]

This is amazing. Very interesting discussion too. Great job and I look forward to seeing where this project leads us.

[Arcitec]

Having looked a bit at the code, it's kind of adorable to see C-like code with manual filesystem I/O calls, manual open/close, etc. It's such a thin wrapper that I wonder how many people will pick it over just using C/C++ directly instead.

But there is big potential if this can run NPM modules. Basically a better-than-Node runtime for Node modules.

One thing that confuses me is this:

- non-async by default - can do blocking calls and not use the event loop
- event loop in JS-land. full control over epoll api

If everything is single-threaded and synchronous, how are you getting such amazing scores in the multi-query benchmark at techempower? Is it simply a result of this library answering really rapidly to each request one after the other? Or is it doing some kind of threading/subprocess spawning to be able to handle many clients at once? Edit: Seems like you do have threading and filesystems modules.

And how do you feel about implementing thread-pools of workers in the way that Node/Deno does it? Where things like "read a file" is a single, async JavaScript call which triggers C++ code that tells a thread to do the work and then comes back with it to JS via the JS event loop? That would be a necessity for porting the more complex, filesystem-based Node modules.

I suspect that this is just a fun hobby project and that features like that are pipe dreams, but perhaps this project could go that far...

I highly suggest fixing your license before this project grows too large, if you have any future plans for that. MIT license is toxic and capitalistic and even allows every corporation to use and abuse this project without giving you a dime, while they enrich themselves through it. It allows them to "use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies" without even contributing their code changes back to you. I suggest dual-licensing it. GPL for general public, and a commercial license for companies. Unless you are ok with free use for literally everyone, in which case I suggest at least using a license that forces companies to contribute code back to you if they modify it... 😉

[billywhizz]

Hi @Bananaman. thanks for the comments and suggestions. I'll be getting back to work on this soon and thinking about next steps at moment.

It won't ever have node.js or npm support out of the box. If people are interested in doing something like that it can be done in userland. The goal for the core is to be very small and simple and low level and easy to understand. This is why I have also not implemented an async filesystem module in C++. There are many different ways to do this and i think would be better done in userland where different libraries can take different approaches and community can decide which ones they find useful. at some point in future, if there is a library that is clearly better than others then we can think about moving it into core/stdlib.

Re. threading - v8 does not allow objects to be shared between isolates so threading involves a lot of serdes overhead and not much difference between using threads and processes other than memory usage. There is a threading module available but it just allows you to launch a completely new isolate in a separate thread and share buffers and do atomic operations between the threads.

The techempower benchmark is not using threads - it launches a process for each core available on the host and the performance is down to highly optimised code and a postgres library written from scratch for the pg wire protocol. everything is done on the event loop inside each process.

re. licensing, i am fine with MIT. it's the most open license as far as I understand and i don't have any issues if someone wants to use this commercialy. as far as i am concerned it is public domain and you can do what you like with it.

[Arcitec]

@billywhizz Your plans sound excellent. I've seen the power of implementing libraries in JS, and your runtime has the tools to create very fast code. I wish you a successful future with this project. It's awesome. Good luck! :-)

[Shonke]

@billywhizz If we use the low-level interface of nodejs, can we do the same? It seems that there isn't much packaging here. The JIT of V8 impressed me.

https://github.com/nodejs/node/blob/3b338cfbe9fd649ab9407b9be3661cbc8a081058/src/tcp_wrap.cc

[billywhizz]

@Shonke i have done various experiments using the low level nodejs bindings over the years but not recently. you should definitely see an improvement over the standard libraries if you want something lower level and not as fully featured. but, afaik, using these bindings is unsupported and internal api's are liable to change. in reality that is probably unlikely though.

the other major issue you have doing network/OS level stuff with node.js is that everything is built on top of libuv. so you are stuck with the decisions that have been made there. libuv and node.js c++ bindings definitely introduce some overhead due to the rather heavy abstractions they layer on top of the system call interface. this is kinda why i wanted something much simpler and more direct. debugging thorny internal node.js isssues is a bit of a nightmare due to the complexity of the code. i also didn't want to have big complex abstractions in c/c++ land and would prefer to do that work in JS where it's easier to understand, debug and refactor.

from what i remember, back in early days of node.js the overhead for calling into C++ from JS was much bigger so i think they made those decisions to do a lot of the complex work in C++ land so they could avoid too many calls across that boundary. that seems to have changed a lot now with massive improvements to v8 across the board.

i did this a long time ago, but it's now way out of sync with current node.js releases. https://github.com/billywhizz/minode

[billywhizz]

@Shonke yes, JIT is very good in v8 now and you can get pretty much same if not better perf than C/C++ depending on what you are doing. the big hit to perf in v8 comes with garbage collection - especially longer lived objects that force mark-sweep collection. so, if you want perf, the goal is to be creating as few objects as possible and retaining them rather than constantly creating new work for GC. much of the overhead built into node.js comes from this as far as i can see. complex class hierarchies and objects being created and destroyed everywhere are not good for perf.

string processing is also another thing that causes big perf issues as strings are immutable in JS and have to be copied rather than modified in place. not sure if that will ever change. but, there are tricks you can do to get the v8 engine to optimize string concatenation and things like that if you are really concerned enough to get into those weeds. there's lots of good info on the v8 blog about various optimization techniques and internals of v8. https://v8.dev/blog

[Shonke]

@billywhizz Does this mean putting more functionality into JavaScript for better performance? Is minode the reason for your just-js attempt?

After checking, I found that there seems to be a lot of Java at the top of the list. As far as I know, there are quite a lot of levels in Java, Are they different?

Thank you very much for your patience

[billywhizz]

@billywhizz Does this mean putting more functionality into JavaScript for better performance? Is minode the reason for your just-js attempt?

not sure i understand the question @Shonke. the JS engine in node.js and just-js is the same engine and provides all the standard features of javascript language. the difference is in what is built on top. just-js is very much about providing a very small set of wrappers around c libraries and system call interfaces and leaving higher level abstractions up to community. node.js is very much a general purpose runtime that is cross platform and there are trade-offs in performance etc. for the extra layers of abstraction you get with node.js or any other high level framework in other languages (e.g. Java/dotnet etc.)

After checking, I found that there seems to be a lot of Java at the top of the list. As far as I know, there are quite a lot of levels in Java, Are they different?

again, not quite sure what you are asking. the java implementations tend to score a little lower on techempower than the C++ and Rust ones but they seem to be catching up all the time. different frameworks in Java have different levels of performance and features which affect each other. in most real world scenarios, you would not see such a gap in performance as you do in techempower as the work being done for each request would likely be much more complex, so i wouldn't take it too seriously when choosing a platform/framework to use - it's more about seeing what the max performance is if you need it, which very few people do in practice. right now, just-js is really just an experiment that i work on for fun. i wouldn't advise using it for anything serious, for now at least.