[sdk] Using kfp v2.0+ not able to add pod labels in kubeflow pipeline workflow

[emilyyujieli]

Hello All,

Environment KFP version: 2.2.0 kubeflow verison: 1.8 KFP SDK version: 2.7.0

Background We are trying to add a pod label to inject istio sidecar in kubeflow pipeline pod then we can use service mesh to do authentication due to project requirement.

Issue If we use kfp v1 and add parameter kubernetes.add_pod_label(task=task,label_key="sidecar.istio.io/inject",label_value="true") in the code, then use KFP SDK v1 to compile the script. label can be injected in related workflow and run successfully. However if we add the same parameter in kfp v2 script and use kfp SDK v2 to compile, l can see the label in compiled file showed as below

platform_spec:
  platforms:
    kubernetes:
      deploymentSpec:
        executors:
          exec-load:
            podMetadata:
              labels:
                sidecar.istio.io/inject: 'true'

However when l try to upload the compiled file and execute it via kubeflow pipeline, the label doesn't be injected in related workflow.

Question 1.If this parameter kubernetes.add_pod_label still available in KFP v2? 2.How can l add label in kubeflow pipeline workflow yaml if using kfp v2 since not able to find any information on kubeflow official doucument?

Regards & thanks!

[daro1337]

Not sure if this help you but as a workaround you can enforce istio injection label on namespace resource

  kind: Namespace
  name: dummy
  metadata:
    labels:
      istio-injection: enabled

then every POD in dummy namespace will be part of service mesh.

https://istio.io/latest/docs/setup/additional-setup/sidecar-injection/#controlling-the-injection-policy

[emilyyujieli]

Hi @daro1337,

Thank you for your suggestion, actually the istio-injection label already existed in namespace level. However it seems when we created kubeflow pipeline it not able to inject istio sidecar automatically, that's why we try to add label manually.

Regards & thanks