Deadlock seen in MiGzOutputStream#close:329 version 1.0.0

[jwilliams-ocient]

Hi guys, first of all major thanks for open sourcing this library! I've been able to significantly improve the I/O performance of my application with this.

I just came across an instance where calling MiGzOutputStream#close hangs indefinitely. A heap dump reveals the thread in question is waiting for the thread pool to shut down.

https://github.com/linkedin/migz/blob/30730b6a85ce5678ed5e961f76bbea43da81bbbf/migz/src/main/java/com/linkedin/migz/MiGzOutputStream.java#L329

Is this a known issue? Is there any reason the ForkJoinPool introduced in 2.0-beta1 prevents this behavior?

Inputs:

• numThreads => 96 (48 core machine)
• outputStream => FileOutputStream("/dev/null")
• bytesIn => ~12GiB
• bytesOut => ~3GiB

[jwilliams-ocient]

The heap dump is rather large, so I'd prefer not to upload it but can capture additional context from the MiGzOutputStream object.