tektoncd / pipeline

A cloud-native Pipeline resource.
https://tekton.dev
Apache License 2.0
8.46k stars 1.77k forks source link

array results as params into another task is not working in v0.54.0 #7546

Open dmitry-mightydevops opened 9 months ago

dmitry-mightydevops commented 9 months ago

Expected Behavior

task generating array:

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: build-kaniko-env-vars
spec:
  workspaces:
    - name: source
      description: The source code files to run build task on
  params:
    - name: working_directory
      type: string
    - name: env_vars
      type: array
      description: environment variables to set during _build-time_.
      default: []
  results:
    - name: build_env_args
      description: list of build-arg args used in kaniko args

  steps:
    - name: set-env-vars
      image: bash:latest
      args:
        - '--env-vars'
        - '$(params.env_vars[*])'
      script: |
        #!/usr/bin/env bash
        set -eu

        source /etc/profile.d/helpers.sh
        echo -n "[\"hello\",\"world\"]" | tee $(results.build_env_args.path)
        #echo "[\"--build-arg=ENV=xxx\",\"--build-arg=PYTHON_VERSION=3.9.3\"]" > $(results.build_env_args.path)
        echo "OK"
        cat $(results.build_env_args.path)

task that should consume the output array as env_vars param input

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: build-kaniko
spec:
  workspaces:
    - name: source
      description: The source code files to run build task on
  params:
    - name: working_directory
      type: string
    - name: builder
      description: The image on which builds will run (default is v1.5.1)
      type: string
    - name: source_subpath
      description: a subpath within the `source` input where the source to build is located.
      default: ''
    - name: dockerfile
      description: Path to the Dockerfile to build.
      default: ./Dockerfile
    - name: extra_args
      type: array
      default: []
    - name: app_image
      description: the name of where to store the app image.
    - name: env_vars
      type: array
      description: environment variables to set during _build-time_.
      default: []
  results:
    - name: app_image_digest
      description: the digest of the built `app_image`.
    - name: app_image_url
      description: the url of the built `app_image`.

  steps:
    - name: build
      image:  $(params.builder)
      args:
        - $(params.extra_args)
        - $(params.env_vars)  <-------------------------------- THIS
        - --dockerfile=$(params.dockerfile)
        - --context=$(params.source_subpath)
        - --destination=$(params.app_image)
        - --digest-file=$(results.app_image_digest.path)

portion of the pipeline that uses the results as params to another task:

  # ⚫ Build kaniko
    - name: build-kaniko-env-vars
      when:
        - input: $(params.workflow.build)
          operator: in
          values: ['true']
        - input: $(params.build.use)
          operator: in
          values: ['kaniko']
      taskRef:
        name: build-kaniko-env-vars
      runAfter:
        - pre-build
      workspaces:
        - name: source
          workspace: source
      params:
        - name:  working_directory
          value: $(params.git.checkout_directory)
        - name:  env_vars
          value: $(params.build_env_vars[*])

    - name: build-kaniko
      when:
        - input: $(params.workflow.build)
          operator: in
          values: ['true']
        - input: $(params.build.use)
          operator: in
          values: ['kaniko']
      taskRef:
        name: build-kaniko
      runAfter:
        - build-kaniko-env-vars
      workspaces:
        - name: source
          workspace: source
      params:
        - name:  working_directory
          value: $(params.git.checkout_directory)
        - name:  builder
          value: $(params.build.builder)
        - name:  dockerfile
          value: $(params.build.dockerfile)
        - name:  source_subpath
          value: $(params.build.source_subpath)
        - name:  app_image
          value: $(params.build.app_image)
        - name:  extra_args
          value: $(params.build_args[*])
        - name:  env_vars
          value: $(tasks.build-kaniko-env-vars.results.build_env_args[*])

Actual Behavior

kg taskrun buildpack-pipeline-build-kaniko -o yaml
   ...
    steps:
    - args:
      - --destination=111.dkr.ecr.us-east-1.amazonaws.com/111/prod/backend:latest2
      - --destination=111.dkr.ecr.us-east-1.amazonaws.com/111/prod/backend:latest3
      - --dockerfile=./Dockerfile
      - --context=docker/python-flask
      - --destination=111.dkr.ecr.us-east-1.amazonaws.com/111/prod/backend:fca45e
      - --digest-file=/tekton/results/app_image_digest
      computeResources: {}
      env:
      - name: source_subpath
        value: docker/python-flask
      - name: dockerfile
        value: ./Dockerfile
      - name: app_image
        value: 111.dkr.ecr.us-east-1.amazonaws.com/111/prod/backend:fca45e
      image: gcr.io/kaniko-project/executor:v1.19.2-debug

however I would expect to see 2 values in the step args:

Steps to Reproduce the Problem

enable-api-fields is set to beta

# Setting this flag will determine which gated features are enabled.
# Acceptable values are "stable", "beta", or "alpha".
enable-api-fields: "beta"

Additional Info

l-qing commented 8 months ago

This is because your result build_env_args wasn't declared as an array type, it was treated as a string type.

If you add the declaration of an array type, the execution pipeline can yield the result you want.

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: build-kaniko-env-vars
spec:
  params:
    - name: working_directory
      type: string
    - name: env_vars
      type: array
      description: environment variables to set during _build-time_.
      default: []
  results:
    - name: build_env_args
      type: array            # <--------------- THIS
      description: list of build-arg args used in kaniko args

  steps:
    - name: set-env-vars
      image: ubuntu
      args:
        - '--env-vars'
        - '$(params.env_vars[*])'
      script: |
        #!/usr/bin/env bash
        set -eu

        # source /etc/profile.d/helpers.sh
        echo -n "[\"hello\",\"world\"]" | tee $(results.build_env_args.path)
        #echo "[\"--build-arg=ENV=xxx\",\"--build-arg=PYTHON_VERSION=3.9.3\"]" > $(results.build_env_args.path)
        echo "OK"
        cat $(results.build_env_args.path)
---
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: build-kaniko
spec:
  params:
    - name: working_directory
      type: string
    - name: builder
      description: The image on which builds will run (default is v1.5.1)
      type: string
    - name: source_subpath
      description: a subpath within the `source` input where the source to build is located.
      default: ''
    - name: dockerfile
      description: Path to the Dockerfile to build.
      default: ./Dockerfile
    - name: extra_args
      type: array
      default: []
    - name: app_image
      description: the name of where to store the app image.
    - name: env_vars
      type: array
      description: environment variables to set during _build-time_.
      default: []
  results:
    - name: app_image_digest
      description: the digest of the built `app_image`.
    - name: app_image_url
      description: the url of the built `app_image`.

  steps:
    - name: build
      image:  $(params.builder)
      args:
        - $(params.extra_args)
        - $(params.env_vars)
        - --dockerfile=$(params.dockerfile)
        - --context=$(params.source_subpath)
        - --destination=$(params.app_image)
        - --digest-file=$(results.app_image_digest.path)
---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: build-kaniko-x
spec:
  pipelineSpec:
    tasks:
      # ⚫ Build kaniko
      - name: build-kaniko-env-vars
        taskRef:
          name: build-kaniko-env-vars
        params:
          - name:  working_directory
            value: .
          - name:  env_vars
            value: []
      - name: build-kaniko
        taskRef:
          name: build-kaniko
        runAfter:
          - build-kaniko-env-vars
        params:
          - name:  working_directory
            value: .
          - name:  builder
            value: ubuntu
          - name:  dockerfile
            value: .
          - name:  source_subpath
            value: .
          - name:  app_image
            value: .
          - name:  extra_args
            value: []
          - name:  env_vars
            value: $(tasks.build-kaniko-env-vars.results.build_env_args[*])
$ kubectl get tr build-kaniko-x-build-kaniko -o yaml

  taskSpec:
    params:
    - name: working_directory
      type: string
    - description: The image on which builds will run (default is v1.5.1)
      name: builder
      type: string
    - default: ""
      description: a subpath within the `source` input where the source to build is
        located.
      name: source_subpath
      type: string
    - default: ./Dockerfile
      description: Path to the Dockerfile to build.
      name: dockerfile
      type: string
    - default: []
      name: extra_args
      type: array
    - description: the name of where to store the app image.
      name: app_image
      type: string
    - default: []
      description: environment variables to set during _build-time_.
      name: env_vars
      type: array
    results:
    - description: the digest of the built `app_image`.
      name: app_image_digest
      type: string
    - description: the url of the built `app_image`.
      name: app_image_url
      type: string
    steps:
    - args:
      - hello            # <------- THIS
      - world            # <------- THIS
      - --dockerfile=.
      - --context=.
      - --destination=.
      - --digest-file=/tekton/results/app_image_digest
      computeResources: {}
      image: ubuntu
      name: build