famzah / popen-noshell

A much faster popen() and system() implementation for Linux
68 stars 13 forks source link

Exec() a wrapper program which will then apply any actions needed prior to exec()'ing the real program #13

Closed famzah closed 7 years ago

famzah commented 7 years ago

A fellow developer suggested that we could greatly simplify the critical executed code in the child process by using an intermediate "wrapper" program which would set up pipe redirection, etc. and finally call the real target binary that we wanted to exec().

famzah commented 7 years ago

@nicowilliams, this idea is so much UNIX-like that I couldn't resist to try it.

Unfortunately, executing an intermediate "wrapper" program adds about 50% overhead in the best case.

Here are my tests results:

famzah@vbox64:~/popen-noshell/performance_tests/wrapper$ ./run-tests.sh 

== tiny2 ==

real    0m4.409s
real    0m4.370s
real    0m4.313s

== wrapper-asm ==

real    0m6.392s
real    0m6.364s
real    0m6.424s

== wrapper-gcc-static ==

real    0m14.342s
real    0m14.255s
real    0m14.469s

== wrapper-gcc-std ==

real    0m22.742s
real    0m21.923s
real    0m22.359s

== wrapper-musl-gcc-static ==

real    0m6.910s
real    0m6.990s
real    0m6.879s

== wrapper-musl-gcc-std ==

real    0m11.946s
real    0m12.129s
real    0m11.879s

It's a case-per-case story whether this statistical 50% overhead is a problem in terms of real CPU cycles. But for the sake of the purpose of this library, it's a lot of overhead.

On the bright side, I finally tried musl libc and it's performance is impressive! Only about 8% slower than the pure assembler implementation of this simple execve() wrapper. The size of the generated binaries is also very small:

-rwxrwxr-x 1 famzah famzah    504 Apr 30 23:42 wrapper-asm
-rwxrwxr-x 1 famzah famzah 803392 Apr 30 23:42 wrapper-gcc-static
-rwxrwxr-x 1 famzah famzah   6248 Apr 30 23:42 wrapper-gcc-std
-rwxrwxr-x 1 famzah famzah  17688 Apr 30 23:42 wrapper-musl-gcc-static
-rwxrwxr-x 1 famzah famzah   5632 Apr 30 23:42 wrapper-musl-gcc-std

I had the intent to try LLVM clang, too. But since the assembler code adds about 50% overhead, and the "musl libc" implementation is as fast as the assembler, I don't see any reason to try LLVM clang as well. Nothing runs faster than assembler here.