Open mat0pad opened 3 years ago
@mat0pad Do you have use case for this issue? This is the design of kubeflow UI. which need to import one object and render graph based on it, I am not say it is not doable. but much more complex. That's why we pushed tekton community to support pipelineSpec and taskSpec for custom task feature.
Hi @mat0pad, do you have a use case where you need to use pipelineRef and taskRef? We are only planning to support taskRef for recursion use case only. The KFP-Tekton UI is mainly for rendering generated pipelines from the KFP-Tekton SDK.
Hi @fenglixa @Tomcli I see multiple use cases and benefits for this:
I come from IBM (CIC) and on one of our larger projects we're actually considering moving to our own solution with tekton, tekton dashboard, knative and an artifact storage service because of the lacking feature.
Thanks @mat0pad, I think your use case is more suited for the Tekton dashboard. Kubeflow Pipeline dashboard is fundamentally different because all the information in the dashboard except pods and events are pulled from a relational database where we archive all the Tekton pipelineRun and metadata. This way we are not limited to the Kubernetes etcd service and able to store tens of thousands of jobs, experiments, and pipelines without adding too much overhead to the Kubernetes API.
Please take a look at the KFP design over here. The fundamental idea of KFP is allow users to design their pipelines using the Python SDK and version control the task using a component.yaml
. This way there's very minimum interaction with etcd and users are able to carry their pipelines and tasks across multiple clusters.
https://www.kubeflow.org/docs/components/pipelines/overview/pipelines-overview/#architectural-overview
This also applies to the Argo backend where the Kubeflow Pipeline dashboard will not render the Argo workflowTemplate CRDs.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
/kind feature
Description: The Kubeflow dashboard fails to render all custom CRDS using pipelineRef and taskRef. This makes it differcult to reuse existing Tasks or Pipelines created on the cluster.
Goal: Allow rendering of dashboard graph when using pipelineRef and taskRef