beacon-biosignals / K8sClusterManagers.jl

A Julia cluster manager for Kubernetes
Other
31 stars 5 forks source link

Allow configuration option to not pass worker stdout/stderr back to manager #91

Open ericphanson opened 2 years ago

ericphanson commented 2 years ago

I.e. I think it would be great if the manager could store a Bool for whether or not we want to run these lines:

https://github.com/beacon-biosignals/K8sClusterManagers.jl/blob/905c995d0ca6300a188e2e8dc7ef5c87a7548295/src/native_driver.jl#L118-L124

It probably should default to true to match Base and to help when getting started, although I suspect many users will want to turn it off once their projects have reached some level of maturity.

Why would someone want to turn it off? Folks might already have logging setup on pods by default (like we do at Beacon), in which case having the manager's logs be filled by the workers + it's own is redundant and overly verbose, since you can easily get the workers logs by looking at their pod output. Additionally, the way Base uses this IO object is it prints the result, without using the logging system. That means there's no opportunity to customize or direct the output with the logging system.

jrevels commented 2 years ago

+1

xref https://github.com/JuliaLang/Distributed.jl/issues/51 and other linked issues there