argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
14.87k stars 3.17k forks source link

Argo Workflows Multicluster - adopting a standardize ClusterInventory API #11389

Open juliev0 opened 1 year ago

juliev0 commented 1 year ago

Opening this issue here but TBD on whether the implementation should occur directly in this repository, or if it should be kept in a separate repository and outside of the Workflow Controller in the interest of not creating new bugs here.


Argo Workflows can benefit from adopting a standardized ClusterInventory API and expanding its multi-cluster capabilities. By doing so, it can seamlessly support various multi-cluster projects without the need to import different project APIs. By integrating with a Cluster Inventory API, Argo Workflows can immediately retrieve the status information for a specific cluster. This information can then be used as a templating value within a Workflow, allowing the Workflow to utilize the specific details of that cluster during its execution. For example:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-world-parameters-
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    inputs:
      clusterInventories: # list of cluster inventories
      - cluster1
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.clusterInventories.cluster1.name}}", "{{inputs.clusterInventories.cluster1.allocatableMemory}}"] # cluster inventory templating

In the future, Argo Workflows has the potential to leverage the Cluster Inventory to determine the availability and resource capacities of registered clusters. This valuable information can significantly enhance workflow scheduling, ensuring optimal resource utilization across clusters. Argo Workflow would be able to intelligently distribute tasks or stages of a workflow to different clusters based on their capacity and current workload, further optimizing the overall performance


Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

danmx commented 6 days ago

Any progress on this issue?