search

hydro-project / hydroflow

Hydro's low-level dataflow runtime
https://hydro.run/docs/hydroflow/
Apache License 2.0
477 stars 34 forks source link

hydro_deploy: figure out principled logging mechanisms #1359

Open MingweiSamuel opened 2 months ago

MingweiSamuel commented 2 months ago

Two areas - right now logging has to be done through ProgressTracker::println as otherwise the progress bars will overwrite. This caused an debugging headache for me because the perf command not found line was printed via println! and getting hidden, lol. Would be good to nail down how we do logging in hydro deploy, both for the deploy itself (maybe make it compatible with tracing) as well as a way to handle the logs on the remote machines. Also possible integration with cloud provider's log frameworks, to think about.

MingweiSamuel commented 2 months ago

Could be interesting to integrate tracing layers with ProgressTracker nodes somehow

MingweiSamuel commented 1 day ago

related: #1415