All of our runtime environments (the linux shell, docker, and kubernetes) have powerful tools for manipulating the stdout and stderr streams, but fv3run captures these streams and instead puts them in relatively inaccessible files. This makes real time debugging and monitoring of k8s jobs especially challenging. In general, it is more flexible to pipe the outputs to stderr and stdout and let the user leverage the power of linux shell, pipes, and commands like tee to put the outputs where they want.
will emulate the previous behavior provided the output directory is made
first.
Changes
Avoid redirecting stderr and stdout when using the fv3run CLI unless the --capture-output argument is passed to run_native.
add an optional argument capture_output to run_native. Unlike the CLI, it's default value is True to avoid breaking all the tests.
Remove the run function in __main__. This does not appear to be public function, and it seems like an unnecessary layer of indirection between the argument parsing and the calls to run_native, etc.
All of our runtime environments (the linux shell, docker, and kubernetes) have powerful tools for manipulating the stdout and stderr streams, but fv3run captures these streams and instead puts them in relatively inaccessible files. This makes real time debugging and monitoring of k8s jobs especially challenging. In general, it is more flexible to pipe the outputs to stderr and stdout and let the user leverage the power of linux shell, pipes, and commands like
tee
to put the outputs where they want.As an example
will emulate the previous behavior provided the output directory is made first.
Changes
fv3run
CLI unless the--capture-output
argument is passed torun_native
.capture_output
torun_native
. Unlike the CLI, it's default value is True to avoid breaking all the tests.run
function in__main__
. This does not appear to be public function, and it seems like an unnecessary layer of indirection between the argument parsing and the calls torun_native
, etc.