Isolate the operating environments of different algorithms

[meua]

Is your feature request related to a problem? Please describe. Different algorithms depend on different versions of python and different versions of dependent installation packages. Packages that result in algorithmic dependencies are very prone to conflicts.

Describe the solution you'd like Therefore, it is expected that different algorithms can run in isolation and independently. Reduce the interference between algorithms.

Describe alternatives you've considered Use namespaces or virtual environments to isolate different algorithm running environments

[haixuanTao]

So, for the dora-drives starter project explicitly, we don't want to have too many environments to manage with its own set of dependencies. If you really want to use different environment, you can use the shell command within the dataflow if you want to use different python interpreter with custom node. Ex:

  - id: object_detection
    custom:
      source: shell
      args: conda activate env3.10 && python object_detection_custom_node.py
      inputs:
        image: webcam/image
      outputs:
        - bbox

[meua]

So, for the dora-drives starter project explicitly, we don't want to have too many environments to manage with its own set of dependencies. If you really want to use different environment, you can use the shell command within the dataflow if you want to use different python interpreter with custom node. Ex:

  - id: object_detection
    custom:
      source: shell
      args: conda activate env3.10 && python object_detection_custom_node.py
      inputs:
        image: webcam/image
      outputs:
        - bbox

- id: midas_op
custom:
source: shell
args: conda activate midas-py310 && python ../../operators/midas_op.py
outputs:
- depth_frame
inputs:
image: webcam/image
env:
PYTORCH_DEVICE: "cuda"
MIDAS_PATH: $DORA_DEP_HOME/dependencies/MiDaS/
MIDAS_WEIGHT_PATH: $DORA_DEP_HOME/dependencies/MiDaS/weights/midas_v21_small_256.pt
MODEL_TYPE: "MiDaS_small"
MODEL_NAME: "MiDaS_small"

Hello, I write according to the above, follow the function you described above, or throw the following exception: Logs from midas_op. sh: 1: conda: not found

[meua]

@haixuanTao I need your help.

[haixuanTao]

You can try: conda run ../../operators/midas_op.py -n midas-py310

If you solved the conda path issue. As mentioned above, we do not recommend having several environment as it creates long term complexity and we're not looking to maintain several environmnent.

[meua]

You can try: conda run ../../operators/midas_op.py -n midas-py310

If you solved the conda path issue. As mentioned above, we do not recommend having several environment as it creates long term complexity and we're not looking to maintain several environmnent.

  - id: midas_op
    custom:
      source: shell
      args: /home/jarvis/anaconda3/condabin/conda run -n midas-py310 python ../../operators/midas_op.py
      outputs:
        - depth_frame
      inputs:
        image: webcam/image

The path problem is solved, but there are still the following problems:

  2023-05-24T12:49:40.937803Z  WARN dora_daemon::pending: node `midas_op` exited before initializing dora connection
    at binaries/daemon/src/pending.rs:76

  2023-05-24T12:49:40.984306Z ERROR dora_daemon: 
    f17ed016-1b0d-496b-bc5a-c857e2282773/webcam failed with exit code 1.

    Check logs using: dora logs f17ed016-1b0d-496b-bc5a-c857e2282773 webcam

    at binaries/daemon/src/lib.rs:906

  2023-05-24T12:49:41.001065Z ERROR dora_daemon: 
    f17ed016-1b0d-496b-bc5a-c857e2282773/plot failed with exit code 1.

    Check logs using: dora logs f17ed016-1b0d-496b-bc5a-c857e2282773 plot

    at binaries/daemon/src/lib.rs:906

(midas-py310) jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ dora-cli logs f17ed016-1b0d-496b-bc5a-c857e2282773 webcam
less: /home/jarvis/anaconda3/envs/dora3.7/lib/libtinfo.so.6: no version information available (required by less)
─────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     │ Logs from webcam.
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ Traceback (most recent call last):
   2 │   File "<string>", line 1, in <module>
   3 │ RuntimeError: Dora Runtime raised an error.
   4 │ 
   5 │ Caused by:
   6 │    0: main task failed
   7 │    1: failed to init event stream
   8 │    2: subscribe failed: Some nodes exited before subscribing to dora: {NodeId("midas_op")}
   9 │ 
  10 │       This is typically happens when an initialization error occurs
  11 │                       in the node or operator. To check the output of the failed
  12 │                       nodes, run `dora logs f17ed016-1b0d-496b-bc5a-c857e2282773 <node_id>`.
  13 │ 
  14 │ Location:
  15 │     apis/rust/node/src/event_stream/mod.rs:70:17
─────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
(midas-py310) jarvis@jia:~/coding/dora_home/dora/binaries/cli/src$ 

[haixuanTao]

https://github.com/dora-rs/dora/pull/315 should help with this issue

[meua]

dora-rs/dora#315 should help with this issue

Ok, I'll take the time to try it out, any help is greatly appreciated.