Background worker and multi-worker support.

[hsyed]

From observation compiling a target which has one changed kt target in it generally takes 3-4 seconds on my mac pro. The targets are currently small -- 4-10 kt files. The Kotlin compiler is known to be 20-35% slower so it stands to reason that a background worker should make the Kotlin compiles for simple targets immaterial.

@pcj I have no idea on how to do this, but I bet you have thought about it ;), If you could you outline a design for this maybe I can do some work on this ? I have some free time at the moment.

thoughts:

• Is there some infrastructure from Jetbrains for this -- i.e., in the preloader, or could the shell be used as a background worker ? If not would nailgun make sense ?
• What would it take to tie the lifecycle of the background worker to Bazel --i.e., shutting down bazel should shutdown the worker.

[pcj]

rules_kotlin is already using bazel workers by default which is configured in the action at https://github.com/pubref/rules_kotlin/blob/master/kotlin/rules.bzl#L100. In my previous experiments this improved performance by around 30-40%. You could try disabling this as a sanity check to see if we are still getting similar perf improvements. Removing ABI jars in the most recent updates will increase built time as well, depending on project structure.

[hsyed]

implemented in my fork. might be out of scope for these rules.