brettwooldridge / NuProcess

Low-overhead, non-blocking I/O, external Process implementation for Java
Apache License 2.0
710 stars 84 forks source link

Reduce pipe buffer size on Windows (fixes #117). #118

Closed bturner closed 3 years ago

bturner commented 3 years ago

Using a large buffer size for the pipe connecting the child process can result in IOCP imposing (very short) delays before notifying the port when a read completes, waiting in case additional data arrives. For a small number of reads the short delays are no problem, but for processes that produce hundreds of megabytes of output (especially if they do so in small bursts, like git http-backend and git upload-pack) all those short delays accumulate into a massive performance hit, reducing throughput compared to ProcessBuilder by an order of magnitude (30MB/s+ down to 2MB/s).

Using a smaller pipe buffer does impose some constraint on how much data can be moved by a single ReadFile or WriteFile call but it also prevents delays (because there's no point in waiting for more input if the buffer is already full), resulting in overall improved--and, more importantly, more consistent--performance.

sschuberth commented 3 years ago

Any chance to get this merged @brettwooldridge? The unconditional e.printStackTrace() is highly disturbing.