Substantially decreasing process startup time

[jbenet]

The current cost of spawning processes in the linux kernel has given rise to a many odd side effects when high performance is a concern (so always):

• very expensive to run pipe tool chains
• event loops / green threading as "the right way" to do multiprocessing
• re-writing of large code-bases to make them one-lang libs (clibs), e.g. libgit2
• programs tend to be monolithic: import everything into same process
• large apps/services tend to be in only one language / use FFIs sparingly
• node bypasses all this (modules, require, versioning, isolation)

These all fly in the face of the UNIX principles.

What if we got process spawning to be about as cheap as userland green threads. This sounds absurdly hard given all the traps/verification that currently happens to syscalls. But consider a microkernel, or a fork of linux, that virtualizes the default process spawn to make it live under the same (or virtually the same) userland sandbox. We get great modularity, multiprocessing, and pipes back!

Think Docker but inside the kernel. "you think you're spawning another VM Process, but you're really not! we're just virtualizing that and cleverly isolating it!" (... in a cheaper way than the old, less clever, expensive (and already virtual!) isolation we were doing before...)

This could be a huge win for software engineering.

[jbenet]

@deian tell me why this is stupid/already has been done.

[jbenet]

@substack, you might like this

[jbenet]

Processes extremely cheap

http://docs.huihoo.com/plan9/Plan9.pdf

[groundwater]

I've been looking at using v8 isolates to start "child node processes" instead of a new kernel processes. The biggest reasons for that are not performance however, but that it provides complete isolation of your child process.

There may be a performance gain because you're not loading a new copy of v8 into memory, but I think the truth is in the nonexistent benchmark.