dimo414 / bkt

a subprocess caching utility, available as a command line binary and a Rust library.
https://www.bkt.rs
MIT License
251 stars 13 forks source link

Stream output while subprocess is running #43

Closed dimo414 closed 1 year ago

dimo414 commented 1 year ago

This PR reimplements Bkt::execute_subprocess() and its callers to support streaming the subprocess' output while it's running, rather than blocking silently until the subprocess finishes.

As suggested in https://stackoverflow.com/q/66060139 the child process' streams are processed in background threads responsible for both persisting and streaming the output to the caller's stdout/stderr.

Scoped threads are used to support streaming to references such as &mut Vec<u8> and to loosen the type requirements of the stream (namely to not require that they are 'static).

CLI:

Library:

Fixes #31

dimo414 commented 1 year ago

Moving this out of the PR description before merging:

I'm positing this as a PR to solicit feedback before merging, so please feel free to weigh in on any parts of this change you think could be improved. Some areas for potential feedback:

  • The signatures and ergonomics of the affected and new methods
    • In particular the impl Write+Send trait bound (which prevents us from using StdoutLock/StderrLock because they are not Send)
  • The use of scoped threads to avoid the 'static trait bound
  • Error handling and Result propagation
    • Notably, currently the .join() calls are .expect()-ed because they don't seem to be compatible with ?
  • The DisregardBrokenPipe behavior, and whether it should be moved into the library core