Reduce overhead introduced by routing every Ninja command through Craftr

[NiklasRosenstein]

I did some benchmarking on my system between Craftr 2 and Craftr 3 with the Cinema 4D SDK (146 compile steps + 1 link step). Because every system command that is invoked by Ninja is routed through the Craftr 3 CLI, it introduces a lot of overhead.

Craftr 2	Craftr 3
26.215s	50.240s

That's a pretty terrible result. 😅

[NiklasRosenstein]

I've been experimenting with a possible solution to the problem. There are two separate overheads involved: A linear time overhead of loading all Node.py components for the craftr program, and a quadratic overhead of reading the craftr_build_graph.json for every command that is invoked by Ninja (the bigger the build graph, the more commands there are to run).

To address the build graph IO issue, the Craftr master process now runs a server that instead serves only the information required for a single action via a TCP connection.

Surprisingly, this didn't make any difference in my test case, which my be still too small (357 build commands). I do think that for now this solves a previous critical scalability problem, though.

Note: This is a different test case with 357 build commands

Craftr 3	with ActionServer
1m27s	1m26s

(0ac08acb2bdc6f1eca83c7021497e63f0d0b5489)

The other issue, that a big portion of Craftr (which always includes the Node.py runtime) needs to be loaded before the action can be run (because all commands are run through the craftr program) actually adds a big chunk of extra time to every command that Ninja invokes. To address this, I have added a separate script that is run via vanilla Python which executes the build action. This actually cuts the build times to 2/3, which is quite a noticable improvement.

Craftr 3	with vanilla Python build slave
1m27s	53s

(f2f7343)

Possible solutions/improvements for the future

1. Serialize every build action into a shell script, and have Ninja run that shell script
2. No slave process -- have Ninja perform a simple command to tell the master process to perform a build and stream the process' output via the TCP socket (possible bandwidth problem? how to separate stdout/stderr?)
3. Further decrease overhead of slave process -- eg. by implementing a native executable in C/C++

[NiklasRosenstein]

	Craftr 2	Craftr 3 (new)	Craftr 3 (previous)
Build Commands	370	357	357
Time	43s	53s	1m27s
Percent	100%	127%	209%

[NiklasRosenstein]

Added a ninja:speed option that can be set to true in which case shell scripts are exported for every build rule and the Build server and client are not used and no routing through the Python client is required.

Craftr 2	Craftr 4	Craftr 4 (ninja:speed=true)
29.05s	35.07s	29.8s