Test newer GC options

[senderista]

We are currently seeing GC pauses of 1-3 minutes on every worker for large joins (> 1B rows), using the default Parallel/Throughput collector. A newer collector like G1 (which is designed for > 4GB heaps like the ones in Myria production) should decrease these pauses significantly, though how much remains to be seen. Though arguably we're targeting throughput rather than latency, pauses of more than a few seconds negatively impact network and other timeouts, causing unnecessary retries and errors.

• [ ] Complete #769
• [ ] Benchmark CMS vs G1 on a variety of workloads
• [ ] Determine options for preferred GC

[senderista]

Assigning to @jingjingwang since she's been hacking the JDK heap management code.

[senderista]

Assuming that G1 is the only reasonable GC choice after moving to Java8, we should measure system throughput and error rate for various choices of MaxGCPauseMillis (default is 200ms).

[senderista]

We've already moved to Java8/G1 after the REEF port was merged, but haven't done any benchmarking or tuning. This article suggests the following options for Spark workloads on JVM8 with G1:

-XX:InitiatingHeapOccupancyPercent=35 -XX:ConcGCThread=20