Add optional ability to use Turbine

[treblereel]

Turbine is a header compiler for Java that allows us to reduce the source code to parse, leading to decreased compile time. By ignoring changes in the body, we can also avoid downstream builds that can be skipped.

This feature is optional for now, but may be set to the default in a future release. To enable this, assign the taskMappings key of stripped_bytecode_headers to the value turbine.

Co-authored-by: Colin Alworth colin@vertispan.com Fixes #150

[treblereel]

i did few perf test on my mac mini m1 16gb:

i used my crysknife demo that contains elemental2-* and many gwtproject modules including heavy like gwt-widgets

with turbine:

(base) treblereel@Dmitiis-Mac-mini demo % hyperfine --runs 10 'mvn clean package' Benchmark 1: mvn clean package Time (mean ± σ): 97.681 s ± 1.508 s [User: 356.236 s, System: 268.646 s] Range (min … max): 95.731 s … 100.939 s 10 runs

without :

Benchmark 1: mvn clean package Time (mean ± σ): 103.222 s ± 0.706 s [User: 371.494 s, System: 274.101 s] Range (min … max): 102.359 s … 104.468 s 10 runs

as @niloc132 can see, tubine is slightly faster, the main bottle neck is ADVANCED comp mode.

So i think, adding turbine makes sense only if we want to have incremental mode. The good news, turbine isn't slower.

[niloc132]

Excellent - I wasn't hoping for performance improvements here, but that this would instead make non-clean builds faster with only some modules changing.

[treblereel]

@niloc132 few notes:

• https://github.com/google/turbine/blob/dd578b81d048aff7bab4d4772eb37ef11d5cd3b2/java/com/google/turbine/options/TurbineOptions.java#L153 expects us to provide set of jars, so for compileClasspath i provide -stripped_bytecode_headers/results/output.jar Maybe we should tweak turbine to allow it to process raw .class files

• https://github.com/google/turbine/blob/dd578b81d048aff7bab4d4772eb37ef11d5cd3b2/java/com/google/turbine/options/TurbineOptions.java#L34 should be investigated

• and, i know how you like it, we have issues with apt again. If apt is presented in reactor, turbine can't process if because it can't find all deps ... I skip this exception for now.

[niloc132]

As long as you are only focusing on the stripped-bytecode and stripped-bytecode-headers, we don't care about APT, that is already finished. It would make sense later to follow up and work on the APT also, but that's an earlier phase of the build, and we're still using javac for that.

If this task fails, whether turbine or javac, we can safely ignore it, since APT isn't necessary (and we can't tell which dependencies are apt-only, which are actual runtime deps).

--

if jars are not optional, we can either tweak how j2cl runs to use that jar (assuming that the jar is 100% deterministic...), or we can add a "make a jar before compiling" like how you unzip afterwards. Not pretty, but it will work, and probably still worth it performance-wise.

Definitely should consider patching turbine to support just reading off the filesystem though.

[niloc132]

Update from discussion in gitter: we're going to have this feature optional (but disabled by default in 0.20), and require java11 rather than add the java version check.

Starting in 0.21, the tooling will require java11, since closure compiler is also dropping java8 support. Turbine will likely be enabled by default starting in 0.21.