search

dora-rs / dora

DORA (Dataflow-Oriented Robotic Architecture) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dataflow capabilities. Applications are modeled as directed graphs, also referred to as pipelines.
https://dora-rs.ai
Apache License 2.0
1.5k stars 79 forks source link

Fix CI build errors #491

Closed phil-opp closed 5 months ago

phil-opp commented 5 months ago

We noticed some CI build errors in https://github.com/dora-rs/dora/pull/489.

phil-opp commented 5 months ago

@haixuanTao I feel like I'm close to some possible solution here. With the latest commit I'm getting the following error on Linux/macOS:

 📦 Built wheel for abi3 Python ≥ 3.7 to /home/runner/work/dora/dora/target/wheels/dora_rs-0.3.3-cp37-abi3-manylinux_2_34_x86_64.whl
Processing ./target/wheels/dora_rs-0.3.3-cp37-abi3-manylinux_2_34_x86_64.whl
Collecting pyarrow
  Downloading pyarrow-16.0.0-cp310-cp310-manylinux_2_28_x86_64.whl (40.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 MB 53.4 MB/s eta 0:00:00
Collecting numpy>=1.16.6
  Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 90.6 MB/s eta 0:00:00
Installing collected packages: numpy, pyarrow, dora-rs
Successfully installed dora-rs-0.3.3 numpy-1.26.4 pyarrow-16.0.0
Created new Python operator `op_1` at ./test_python_project/op_1
Created new Python operator `op_2` at ./test_python_project/op_2
Created new Python node `node_1` at ./test_python_project/node_1
Created new yaml dataflow `test_python_project` at ./test_python_project
Traceback (most recent call last):
  File "<string>", line 2, in <module>
ModuleNotFoundError: No module named 'dora'
Dataflow could not be validated.

Do you know what could cause this ModuleNotFoundError?

haixuanTao commented 5 months ago

The python being called is not the one from the virtual env where dora-rs is installed. By default, it is using the one from the dora-daemon which has been spawned before activating the virtual env. I would recommend either:

dora destroy 
dora up

Or changing the order in which the installation is done.

This is actually very annoying as it can be a bit confusing. But I don't know if there is much we can do now.

I think in the future we should probably do something like

dora run

With ephemeral daemon, and only use 

dora up

When we're in a distributed environemnt

phil-opp commented 5 months ago

Thanks a lot for the explanation!

I split up the CLI job into a Rust job and a Python job and used separate dora up commands.

I think in the future we should probably do something like

dora run

Good idea!