benchmark execline init scripts

[dreamcat4]

Laurent says:

Use execline scripting for init, because init only needs to be run 1 time at system start. However this is not true for containers (or jails, namespaces, or whatever you want to call it). Which uses linux namespaces kernel feature.

The startup time is OK if you have just a few containers. Then it OK and does not seem too slow.

However it is possible to have many more containers running. For example on a production server - to utilize the full host resources.

Then the startup time matter more. For example, let us imagine a XEON host was fully loaded, and it had to start up 100 containers at boot time. If we assume execline based scripts incur a 1 second delay starting inside each container. Then in total that take 100 seconds (additional overhead). For to bring up the server or restart it.

I have not tested yet our init startup time. It is not "1 second". Yet it is not instant.

Maybe in future (eventually) we can write a 'C' program instead to replace the stage1 and stage2. Then such extra delay / overhead can be a lot less. Expecially if not launching out many individual seperate programs (like we are during the execline script).

[glerchundi]

I don't agree with you on this.

We didn't make any benchmarking on the execline script. Sure execline will not perform as well as an init done in 'C' but it is vastly more maintainable. If I had to put both on a scale, i would choose execline, this is a clear example of premature optimization.

I'm going to rename this issue to reflect what we first really need.

[dreamcat4]

OK. I have run simple benchmark with time command. In the myprogram.sh it just does this only:

#!/bin/sh
exit 0

Gist of the full results terminal output: https://gist.github.com/dreamcat4/1e802f322a1599ef4c14

Unfortunately I could not seperate the s6 startup time from the shutdown time. So This is the total time, including both the s6 startup + shutdown.

Results Summary:

These numbers are the average od 5 indentical runs. On a Core2Duo T7200 2.0GHz, with 4GB ram. Ubuntu 14.10 with a single 7200RPM HDD.

 no s6: 1.8666 sec
old s6: 3.8896 sec
new s6: 6.9312 sec

There is not much meaningful we can say yet. As this still seems early versions. But I hope it demonstrates the general point.

I think we can continue to fix bugs for now. And come back to this matter later on. There are 2 approaches. My initial suggestion - was to replace entirely with C. Would be a re-write in another language. (which you disagree).

The other approach is staying with execline forever, and gradually try to improve the speed with optimizations or finding shortcuts in the existing execline scripts and codebase. Then if the execline hits a wall, and cannot further be optimized. Then the slowest parts or pieces of it can gradually be converted / replaced from being written in execline in to a compiled sub-program... Which I guess already happens when Laurent makes a new program to replace some parts of execline script. But those changes are usually functionality - focused (not performance focused).

Anyway, we should benchmark again after another round of improvements / fixes. It should become faster than this. This early code seems to be too buggy to make an reasonable judgement just yet.

[dreamcat4]

Seems faster now :)

version  | minimum start+shutdown time
--------------------------------------
-          1.8666 sec
1.4.0      3.8896 sec
1.7.0      6.9312 sec
1.7.1      1.9554 sec

[glerchundi]

hehe, that's great :-)

1.7.0 = 6.9312 sec, this measure surprises me a little bit, have you used S6_KILL_GRACETIME=0: docker run --rm -e S6_KILL_GRACETIME=0 image?

[dreamcat4]

@glerchundi No. Was not aware of that until you pointed it out to me. I be sure to try with that setting for any future time(s) is re-run.

[glerchundi]

I benchmarked both with 1000 samples. Our image under test starts every s6 services up and then executes test -1. Ubuntu does the same with the difference that it doesn't use a s6 supervision tree, take it with a grain of salt but after setting S6_KILL_GRACETIME=0 and removing s6-sleep -m 200 it should be more or less accurate.

ubuntu:14.04:

#> time docker run -ti -e S6_KILL_GRACETIME=0 ubuntu test -1

real   0m0.656s
user   0m0.143s
sys    0m0.008s

justcontainers/base (based in s6-overlay-builder 1.8.1):

#> time docker run -ti -e S6_KILL_GRACETIME=0 base test -1

real   0m0.743s
user   0m0.142s
sys    0m0.008s

So, the results shows that s6 takes more or less 87ms more than its base image starting everything up. Taking into account what we're doing it's quite fast :-)