When a command is executed, all it stdout and stderr output is stored in a buffer in memory. The buffer is used when evaluating the command output and to print it if the command execution should fail. This can be very memory intensive when executed tasks generate a lot of output. To reduce the memory footprint: - only up to 16KB of the beginning and up to 16KB of the end of the output is stored per stdout and stderr for the eventual error message if the command execution fails. The missing part is marked in the error message. This should be enough to understand failures and their context. This is implemented with a modified copy of the prefisuffixsaver from the os/exec stdlib package. - The Run() function does not store and return the whole output of a command on success anymore. Run() is used to execute the commands of baur tasks. This will reduce the memory usage. - The new RunCombinedOut() stores the stdout output in a buffer. This function is used to run baur internal commands that only generate few output and the output is required for evaluation.
running commands can now be canceled via a passed context. This is not used by anything yet. It will be used to specify timeouts for "baur run".
Cmd now supports to specify an additional io.Writer to stream the stdout and stderr output of a command. This enables efficient processing of larger outputs and will be used in a following Pull-Request, to run "git ls-files" to obtain input file digests.
when the command output is streamed to the terminal (baur run --show-task-output), the stderr output is now colored red, that makes it distinguishable from stdout output
when --show-task-output is not passed to baur run and the execution of a command fails, the stderr and stdout output will now be shown after each other instead of intermixed.
When a command is executed, all it stdout and stderr output is stored in a buffer in memory. The buffer is used when evaluating the command output and to print it if the command execution should fail. This can be very memory intensive when executed tasks generate a lot of output. To reduce the memory footprint: - only up to 16KB of the beginning and up to 16KB of the end of the output is stored per stdout and stderr for the eventual error message if the command execution fails. The missing part is marked in the error message. This should be enough to understand failures and their context. This is implemented with a modified copy of the prefisuffixsaver from the os/exec stdlib package. - The Run() function does not store and return the whole output of a command on success anymore. Run() is used to execute the commands of baur tasks. This will reduce the memory usage. - The new RunCombinedOut() stores the stdout output in a buffer. This function is used to run baur internal commands that only generate few output and the output is required for evaluation.
running commands can now be canceled via a passed context. This is not used by anything yet. It will be used to specify timeouts for "baur run".
Cmd now supports to specify an additional io.Writer to stream the stdout and stderr output of a command. This enables efficient processing of larger outputs and will be used in a following Pull-Request, to run "git ls-files" to obtain input file digests.
when the command output is streamed to the terminal (baur run --show-task-output), the stderr output is now colored red, that makes it distinguishable from stdout output
when --show-task-output is not passed to baur run and the execution of a command fails, the stderr and stdout output will now be shown after each other instead of intermixed.
Closes https://github.com/simplesurance/baur/issues/328