[backend] Cannot inject environment variables and volumes into components with the KFP CLI

[b-feldmann]

Environment

• How did you deploy Kubeflow Pipelines (KFP)? Full Kubeflow Deployment

• KFP version: 1.7.0 (we are using Kubeflow 1.7.0)

• KFP SDK version: 2.0.1

Steps to reproduce

The injection of environment variables and secrets into components is not working correctly with kfpv2. Please look below for the python code and the generated yaml.

We could use volumes and environment variables in KFPv1 but when updating to kfpv2 and Kubeflow 1.7 the (updated) code does not work anymore. The Pipeline yaml looks fine from our perspective but in Kubernetes the pipeline component pod does not get the required configurations. (volume mount, env)

Expected result

The Kubernetes Pod has access to the volumes and environment variables and we can use them inside a Kubeflow pipeline component.

Materials and Reference

We are using this to add the secret in the kfp cli.

def add_gsc_access(task):
    task.set_env_variable(
        name="GOOGLE_APPLICATION_CREDENTIALS", value="/var/secrets/google/key.json"
    )

    kubernetes.use_secret_as_volume(
        task,
        secret_name="pubsub-key",
        mount_path="/var/secrets/google",
    )

Generated Pipeline YAML:

components:
  comp-download-from-gcs:
    executorLabel: exec-download-from-gcs
    inputDefinitions:
      parameters:
        gcs_path:
          parameterType: STRING
    outputDefinitions:
      artifacts:
        data:
          artifactType:
            schemaTitle: system.Artifact
            schemaVersion: 0.0.1
...
deploymentSpec:
  executors:
    exec-download-from-gcs:
      container:
        command:
        - ...

---

Impacted by this bug? Give it a 👍.

[X-Siro]

Similar issue with container_component and kubernetes.use_secret_as_env

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.