piotrmurach / tty-command

Execute shell commands with pretty output logging and capture stdout, stderr and exit status.
https://ttytoolkit.org
MIT License
400 stars 34 forks source link

Add ability to stream binary data on stderr and stdout outputs #34

Open piotrmurach opened 6 years ago

piotrmurach commented 6 years ago

As suggested by @janko-m, this library would benefit from ability to stream binary data chunk by chunk to stdout/stderr when available from the running process.

I didn't want to first download the whole remote file onto disk, and then start streaming the downloaded file into the response, because then the user would have to wait for the whole file to get downloaded by the server before it can start downloading it from the server (which isn't ideal if these are videos that you want to play), also it would use unnecessary disk space, even if it will get deleted at the end. I wanted to stream the remote file to the response as it is being downloaded. So I told wget to redirect to stdout. But I couldn't capture a simple string stdout here, because it would mean that the whole file would be loaded into memory. I knew that Open3.popen3 uses Ruby pipes for stdin/stdout/stderr, which are readable/writable IO objects, so after a lot of fiddling I managed to utilize it. It would be great that instead TTY::Command provides some wrapper for that.

jrochkind commented 6 years ago

If such an API is added, it might be easy to also make it give you access to the stdin stream, so you can write things to stdin as you read them from stdout/stderr? Might make it more plausible to use for controlling an interactive command line process.

piotrmurach commented 6 years ago

@jrochkind Thanks for the suggestion! Agree that exposing writable stdin in a block would make total sense. For the time being, there is a way to have interactive input and redirection.

jrochkind commented 6 years ago

I'm not sure having to pre-script the entirety of the stdin counts as 'interactive', but okay!

piotrmurach commented 6 years ago

The example demonstrates providing input in one fell swoop, but I think since you can provide IO-like object, you should be able to write to it when you need to, especially when the process is blocking. So based on the output from stdout/stderr in the block you could write whatever as input.