Long term plan and documentations

[binarycrayon]

this is exciting and would love to build on top of it.

For us, we have 5 data scientists authoring workflow on airflow and I'd like to move them to Argo. (so a good use-case right there). I still need to look into the documentation/code to figure out how this work between airflow/argo

I'm wondering what's the long term plan for this, and what features are considered 1.0 release

[binarycrayon]

OK, I see that Couler currently only supports Argo Workflows.

would love to know more about how the system design will support tekton (not familiar, know it's google/kubeflow) and airflow (familiar)

I'd love to be more involved if possible.

[merlintang]

Currently, kubeflow pipeline also use Argo as the backend.

For the other engine, we can discuss how to proceed.

On Thu, Sep 3, 2020 at 9:17 AM binarycrayon notifications@github.com wrote:

OK, I see that Couler currently only supports Argo Workflows.

would love to know more about how the system design will support tekton (not familiar, know it's google/kubeflow) and airflow (familiar)

I'd love to be more involved if possible.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/couler-proj/couler/issues/34#issuecomment-686174373, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK5R6LSECPRQ6DQWLGN5SLSD3VELANCNFSM4QUB4P7A .

[terrytangyuan]

@binarycrayon Thanks for moving the discussion over here (some context in https://github.com/argoproj-labs/community/issues/3). Yes only Argo Workflow has been open sourced so far and we do plan to support other workflow engines with the help from the community.

There's discussion on the design of the core unified interface in https://github.com/couler-proj/couler/pull/24. We'd appreciate your feedback. If you are interested in helping out with anything in particular, please let us know and we'll go from there.

[dwink]

@terrytangyuan over in the argoproj-labs/community#3 you state:

argo-python-dsl is unnecessary though since Couler provides same functionalities and more well-maintained and will do our best to support Python users from Argo community going forward. Feel free to report issues here if you encounter any problems or missing features.

I was looking for a very clear DSL for Argo-specific YAMLs, which it appears that argo-python-dsl was providing. Can you share how we would generate workflow templates, and support argo sensors and other constructs, in Couler? From what I can see Couler is very limited to submitting workflows ad-hoc, while I am using Argo for event-driven invocations via Argo Events and don't have an interest in ad-hoc submissions.

[terrytangyuan]

Workflow templates are not supported yet. At this projects stands currently, you might want to use argo-client-python for a complete Argo spec. However that project is in need of maintainers, see https://github.com/argoproj-labs/argo-client-python/issues/34.