search

dora-rs / dora

DORA (Dataflow-Oriented Robotic Application) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dataflow capabilities. Applications are modeled as directed graphs, also referred to as pipelines.
https://dora-rs.ai
Apache License 2.0
1.35k stars 67 forks source link

Add `--quiet` flag to daemon and coordinator #548

Closed phil-opp closed 3 weeks ago

phil-opp commented 3 weeks ago

Suppresses all log messages to stdout.

I went with --quiet instead of changing the default and adding a --verbose arg because this even suppresses error output.

haixuanTao commented 3 weeks ago

The thing is within the unix philosophy the default should be quiet and not verbose.

Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input. https://en.wikipedia.org/wiki/Unix_philosophy

Can we make the level filter to ERROR?

haixuanTao commented 3 weeks ago

I think that it is fine that the error message are filtered out, if we want to have it truly quiet.

phil-opp commented 3 weeks ago

I don't think that we should follow UNIX philosophy. The daemon and coordinator are long-running, server-like programs, so nobody will use the output dora daemon as an input to another UNIX tool.

The default filter is WARN and I think that's quite suitable. I don't consider warnings as extraneous information. For example, the coordinator will log a warning if it fails to send a heartbeat message to the daemon. This indicates that something is wrong with the setup, so this isn't something that we should hide by default.

I'm fine with adjusting the log output for the daemon in a follow-up PR. For example, I don't think that we need to print dataflow-related errors and warnings by default because they are already sent back to the CLI. But this is not related to this quiet flag.

phil-opp commented 3 weeks ago

Note that a lot of UNIX commands implement a --quiet option. Examples are grep, make, or git. Some more examples are mentioned here: https://unix.stackexchange.com/questions/191254/why-do-many-commands-provide-a-quiet-option

haixuanTao commented 2 weeks ago

So this merge make it that when we we use dora start without --attach flag and there is an error on initialization there is nothing that is printed out.

We need to somehow reply to dora start in case there is a failure on initialization.

phil-opp commented 2 weeks ago

So the error happens after the dataflow is considered as started an its UUID is returned?

phil-opp commented 2 weeks ago

I opened https://github.com/dora-rs/dora/pull/554 to make it easier to find out whether a dataflow failed.

I don't think that we can return an error on a non-attaching dora start because it directly exits after the dataflow is started.

haixuanTao commented 2 weeks ago

Yeah we could make dora start --attach default in that case and have dora start --detached ?

It's very frustrating to not have any info on errors

haixuanTao commented 2 weeks ago

So the error happens after the dataflow is considered as started an its UUID is returned?

So one error is that I had a python import issue, so at the very beginning of the script before node = Node() and nothing happened.

haixuanTao commented 2 weeks ago

I think we should add CI/CD test on graceful stop for pending nodes.

haixuanTao commented 2 weeks ago

One of the problem with this PR is that when an error happens but the dataflow does not immediately end nothing happens within the terminal which is also very confusing.

If we're going #552, we need to make sure to print also errors that happens before the entire dataflow finishes.

phil-opp commented 2 weeks ago

Good idea! I try to implement this.