search

argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
14.55k stars 3.12k forks source link

Python-Rust binding for API #13205

Open giorgiozoppi opened 1 week ago

giorgiozoppi commented 1 week ago

Summary

Performance: Maturin binding for Python.

Use Cases

When would you use this? We would like reduce the latency in the submission, workflow creation and provide and maintain a API package based on maturin coded in Rust directly to give the ability to submit a lot of workflows at time. Also we want to support a tree structure in the workflows in a simple way. We will use the current API in rust and wrap in python, some experimentation is needed before to see how much we gain.

Message from the maintainers:

Love this feature request? Give it a 👍. We prioritise the proposals with the most 👍.

terrytangyuan commented 1 week ago

We would like reduce the latency in the submission, workflow creation

I am not sure if this is a strong use case. Even if you submit a lot of workflows in the same time, they'll still get queued and the performance is subject to the workflow controller.

giorgiozoppi commented 1 week ago

We aim to reduce the submission latency and the fetching cluster workflow latency, also the monitoring.