search

dora-rs / dora-drives

A step-by-step tutorial that allows beginners to write their own autonomous vehicle program from scratch using a simple starter kit. Dora-drives makes learning autonomous vehicle systems faster and easier.
https://www.dora-rs.ai/docs/guides/dora-drives/
Apache License 2.0
46 stars 11 forks source link

Python first API and dataflow composition #1

Open heyong4725 opened 2 years ago

heyong4725 commented 2 years ago

@haixuanTao currently our dataflow composition is using declarative YAML file. I am wondering in future if we can support Python first SDK such that developer can directly compose the dataflow graph (Python DSL).

Take a example of this very successful project: https://flyte.org/ , https://www.youtube.com/watch?v=OLD5-G9R9fw, https://github.com/flyteorg/flyte, https://github.com/flyteorg/flytekit:

from flytekit import task, workflow

@task(cache=True, cache_version="1", retries=3) def sum(x: int, y: int) -> int: return x + y

@task(cache=True, cache_version="1", retries=3) def square(z: int) -> int: return z*z

@workflow def my_workflow(x: int, y: int) -> int: return sum(x=square(z=x), y=square(z=y))

I am not sure if this type of API style is suitable for Dora, but let's explore it.

haixuanTao commented 2 years ago

There is definitely ideas that we can borrow from this API. Thanks! But, we might have to be cautious to not mix too much imperative programming and declarative YAML programming.

But I think that decorators API for parametrization is going to be a huge plus for developers.