support --ungroup option like GNU-parallel

aaronriekenberg / rust-parallel

Fast command line app in rust/tokio to run commands in parallel. Similar interface to GNU parallel or xargs plus useful features. Listed in Awesome Rust utilities.

MIT License

139 stars 7 forks source link

support --ungroup option like GNU-parallel #21

Open zakstucke opened 3 months ago

zakstucke commented 3 months ago

Love the look of the library.

Looking to replace GNU-parallel with this, but seems like --ungroup isn't an option. In my case, I do want output interleaving as I'm running "infinite" processes, with the current --group behaviour I'll never see those logs come out.

Thanks!

P.s I've just seen in 1.16.0 the behaviour of --exit-on-error was changed to wait for in-process tasks to finish before exiting. Given my intended use case I'd also need the previous behaviour. Maybe 3 separate options needed? I.e.

--exit-on-error -> current behaviour
--kill-on-error-hard -> SIGTERM+SIGHUP all ongoing processes if any error
--kill-on-error-soft -> SIGKILL all ongoing processes if any error

aaronriekenberg commented 3 months ago

@zakstucke you are right the --ungroup option does not exist now. Outputs only happen after each process exits which is similar to the --group behavior in GNU Parallel.

Could you please give an example command line where you want to use --ungroup? Would like to understand more on use case for this.

On the --exit-on-error behavior - you are correct there was a change in version 1.16.0:

In 1.15.0 if --exit-on-error was specified rust-parallel would immediately exit with status 1 after first command fails
In 1.16.0 this behavior changed so on errors rust-parallel waits for in progress commands to finish before exiting with status 1

Could you please write a second issue for more exit-on-error options?

Thanks!

zakstucke commented 3 months ago

@aaronriekenberg my specific use case:

Running multiple processes from a single docker instance, e.g. a webserver + background task processor.
I used to do this using GNU parallel, but for various reasons it's caused me problems, and I wanted a simple binary parallel runner instead, so rust-parallel sounded like a good fit.
In regards to --ungroup, logging is done separately, but some command line stdout/stderr exists and needs to be seen. Given all processes are ideally "endless", this needs to be printed to console straight away with no buffer, otherwise it will be i'm assuming endlessly buffered inside rust-parallel.
In regards to --exit-on-error, as I mentioned above, I need the container to exit if any process dies, but given each process is infinite, unless a termination is sent to the other processes, the docker container will not exit. The option for different kill signals being sent to the healthy processes allows the user to "softly" kill processes with e.g. SIGTERM, or instantly with SIGINT.