dora-rs / dora

DORA (Dataflow-Oriented Robotic Application) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dataflow capabilities. Applications are modeled as directed graphs, also referred to as pipelines.
https://dora-rs.ai
Apache License 2.0
1.35k stars 68 forks source link

Simplify the `run.rs` scripts of our dataflow examples using `xshell` #495

Open phil-opp opened 2 months ago

phil-opp commented 2 months ago

The xshell crate provides a more convenient way to build and run a Command, which is more similar to traditional bash scripts. Using this crate, we can simplify our run.rs scripts while still being platform-independent and not requiring any external dependencies. We can also still run the examples using cargo run --example.

Alternative to #454 .

phil-opp commented 1 month ago

Do we want to proceed with this approach? Personally, I think this is a considerable improvement because it changes the previous Command builder code to something that resembles normal shell commands:

From:

let cargo = std::env::var("CARGO").unwrap();
    let mut cmd = tokio::process::Command::new(&cargo);
    cmd.arg("run");
    cmd.arg("--package").arg("dora-cli");
    cmd.arg("--").arg("build").arg(dataflow);
    if !cmd.status().await?.success() {
        bail!("failed to build dataflow");
    };

To:

let dora = prepare_dora(&sh)?;

cmd!(sh, "{dora} build dataflow.yml").run()?;

Possible alternatives:

haixuanTao commented 1 month ago

Looks good to me !

phil-opp commented 1 month ago

@haixuanTao I get a ModuleNotFoundError: No module named 'dora' error again with the new python-operator-dataflow example. The venv is created before dora up, so it doesn't seem like the same issue as https://github.com/dora-rs/dora/pull/491#issuecomment-2079464273. Any idea what it could be?

phil-opp commented 2 weeks ago

We decided to wait a bit with this PR until https://github.com/dora-rs/dora/pull/551 is ready. We hope that we can create a proper integration test framework then instead of using the example as integration tests. Perhaps we can find a even better solution for running the examples then (after the testing part is moved to the test framework).