search

kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.6k stars 1.62k forks source link

[feature] Add ability to specify custom workflow labels and annotations using KFP V2 #10889

Closed cmdevoto closed 2 months ago

cmdevoto commented 4 months ago

Feature Area

/area backend

What feature would you like to see?

We would like the ability to add custom labels and annotations to our kubeflow pipelines and runs created by them. In KFP V1, we had the option of editing the argo workflow yaml directly to add custom labels and annotations to the object itself. However, with the IR yaml in KFP V2, that is no longer possible and we have no way of supplying additional metadata aside from specifying pod labels and annotations.

What is the use case or pain point?

We use the workflow labels and annotations to identify workflows and store important information about a run. It would be extremely beneficial if the IR yaml had label and annotation fields (potentially under pipelineInfo) and if the argo compiler added these values to the argo workflow yaml.

Is there a workaround currently?

If we specify the custom metadata as pod labels or annotations, the information will be in the argo workflow yaml. However, it is not under metadata.podMetadata and is unclearly nested in the components, making it difficult standardize the retrieval and reassignment of the labels as metadata on the workflow object itself.

Love this idea? Give it a 👍.

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 2 months ago

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.