ai2cm / fv3config

Manipulate FV3GFS run directories
Apache License 2.0
1 stars 0 forks source link

Make output capturing an optional behavior #77

Closed nbren12 closed 4 years ago

nbren12 commented 4 years ago

All of our runtime environments (the linux shell, docker, and kubernetes) have powerful tools for manipulating the stdout and stderr streams, but fv3run captures these streams and instead puts them in relatively inaccessible files. This makes real time debugging and monitoring of k8s jobs especially challenging. In general, it is more flexible to pipe the outputs to stderr and stdout and let the user leverage the power of linux shell, pipes, and commands like tee to put the outputs where they want.

As an example

fv3run fv3config.yml output 2> output/stderr.log > output/stdout.log

will emulate the previous behavior provided the output directory is made first.

Changes

nbren12 commented 4 years ago

I tested this inside of a docker image so it ran all of the integration tests skipped by the CI. It does print the stderr and stdout if wanted.