Output improvements

[bburgin]

1. Added --immediate-output argument, to allow for easier debugging of hung child processes
2. Made output unbuffered, to avoid losing output on os.Exit() call

[shenwei356]

It's a great work! Thank you @bburgin .

[shenwei356]

Hi @bburgin , why didn't you add a new channel for content from stderr (like the Ch for stdout) of command to type Command, to transfer the operations of output of both stdout and stderr to function Run and Run4Output. Do you have any special consideration, for "immediate-output? Is there any way to solve this in higher-level? :thinking:

I test the code, stdout and stderr from few commands sometimes are incomplete when commands failed. This still need some work from us.

[bburgin]

Hi @shenwei356 , since --immediate-output immediately writes the text, I thought the additional channel would be unnecessary. And to reduce layer complexity, it directly writes from logic that is as close to the exec() logic as possible. Is there additional logic you want to run at the higher layers that would require the additional channel?

Can you provide me the test cases that have incomplete output? Then I can debug them. Thanks.

[shenwei356]

Here's a case.

$ seq 10 | ./rush 'ls {}' --immediate-output -o t.txt
[ERRO] wait cmd #4: ls 4: exit status 2
[ERRO] wait cmd #3: ls 3: exit status 2
[ERRO] wait cmd #1: ls 1: exit status 2
[ERRO] wait cmd #2: ls 2: exit status 2
[ERRO] wait cmd #5: ls 5: exit status 2
[ERRO] wait cmd #6: ls 6: exit status 2
[ERRO] wait cmd #7: ls 7: exit status 2
[ERRO] wait cmd #9: ls 9: exit status 2
[ERRO] wait cmd #8: ls 8: exit status 2
[ERRO] wait cmd #10: ls 10: exit status 2
$ cat t.txt
(4/1/1): ls: cannot access '4': No such file or directory
(1/1/1): ls: cannot access '1': No such file or directory
(2/1/1): ls: cannot access '2': No such file or directory
(5/1/1): ls: cannot access '5': No such file or directory
(6/1/1): ls: cannot access '6': No such file or directory
(7/1/1): ls: cannot access '7': No such file or directory
(9/1/1): ls: cannot access '9': No such file or directory
(8/1/1): ls: cannot access '8': No such file or directory
(_2.10/1/1): ls: cannot access '10': No such file or directory
$ cat t.txt | wc -l
9

BTW, why's these a _2. in front of 10?

Running code bellow several times, you will get different line count ranging from 6-10.

seq 10 | ./rush 'ls {}' --immediate-output -o t.txt 2>/dev/null ;  wc -l t.txt

[bburgin]

The _2. in front of 10 allows the user to (after the fact) sort the output and have it in numerical order. So after sorting, the _2.10 appears after 9, instead of after 1.