ok, so this is something I've been trying to get right for years and none of the libraries I found are doing it correctly!
it's based heavily on the implementation in https://github.com/junosuarez/exec-stream/pull/2
the new spawnDuplex stream returns a nodejs stream which is connected to stdin and stdout of a spawned child process.
... and the important bit is that it handles flow-control correctly.
with all other implementations I found, if you have a fast writer and a slow reader then all of the stdout from the writer is buffered in the nodejs memory waiting for the reader to ask for more. this will eventually result in OOM errors.
this implementation uses the SIGSTOP and SIGCONT kill signals to put the child process to sleep when the highwatermark is reached and wakes it up again when the reader is ready for more data 🎉
ok, so this is something I've been trying to get right for years and none of the libraries I found are doing it correctly! it's based heavily on the implementation in https://github.com/junosuarez/exec-stream/pull/2
the new
spawnDuplexstream returns a nodejs stream which is connected tostdinandstdoutof a spawned child process. ... and the important bit is that it handles flow-control correctly.with all other implementations I found, if you have a fast writer and a slow reader then all of the
stdoutfrom the writer is buffered in the nodejs memory waiting for the reader to ask for more. this will eventually result in OOM errors.this implementation uses the
SIGSTOPandSIGCONTkill signals to put the child process to sleep when the highwatermark is reached and wakes it up again when the reader is ready for more data 🎉