Closed g0194776 closed 1 year ago
hi @g0194776 , I guess this is because the value of need-publish-artifact in the result is 'yes'
, not yes
.
as seen in my screenshot, there are more single quotes in the result of task fetch-code.
maybe you could change the when expression to:
when:
- input: $(tasks.fetch-code.results.need-publish-artifact)
operator: in
values:
- "'yes'"
or remove single quote in fetch-code task result.
hope it can help you
@chengjoey Thanks for your reply. That was my first thought, that's why I have to dig into the code by dlv and try to explain the real value in the memory.
That was strange. I can see single quotes only in the Tekton dashboard. The TaskRun output of JSON result and even into the code just see yes
not 'yes'
, I can not explain that.
(dlv) b skipBecauseWhenExpressionsEvaluatedToFalse
Breakpoint 1 (enabled) set at 0x1a831aa for github.com/tektoncd/pipeline/pkg/reconciler/pipelinerun/resources.(*ResolvedPipelineTask).skipBecauseWhenExpressionsEvaluatedToFalse() github.com/tektoncd/pipeline/pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go:437
(dlv) p t.PipelineTask.WhenExpressions
github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1.WhenExpressions len: 1, cap: 1, [
{
Input: "$(tasks.fetch-code.results.need-publish-artifact)", // <-- HERE is the most important thing I can see, the expression not be replaced
Operator: "in",
Values: []string len: 1, cap: 1, ["yes"],}, // <-- HERE, is yes, not 'yes'
]
How can I debug it deeper?
Or was anything else I missed?
@g0194776 ,could you use kubectl
describe you pipelinerun , and see Skipped Tasks
that why the tasks were skipped
@chengjoey
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
annotations:
app.moebius.io/app-definition-id: 63bcdc6cab676200011918e0
app.moebius.io/app-definition-name: 0110测试
app.moebius.io/app-definition-version: v1
app.moebius.io/owner-email: admin@chameleon.io
app.moebius.io/project-id: 6114b9a809e9d4000c06adda
app.moebius.io/project-name: 项目测试0812
app.moebius.io/tenant-id: 6114b8cc0e39e8000145198b
app.moebius.io/tenant-name: xxx
app.moebius.io/trigger-kind: webhook
chameleon.io/plugin-owner: moebius-connector
creationTimestamp: "2023-01-19T04:09:38Z"
generation: 1
labels:
language: go
tekton.dev/pipeline: go
name: 63bcdc7bab676200011918e1-baac20fd-973e4a51
namespace: tekton-pipelines
resourceVersion: "16329554"
uid: ccd04a8d-96e6-4a25-84ea-a6af6de5c9b0
spec:
params:
- name: sonar-project-name
value: git@xxx.git
- name: minio-secret-key
value: xxx
- name: sonar-source-to-scan
value: .
- name: dockerfile
value: ./Dockerfile
- name: cpu-arch
value:
- linux/amd64
- name: artifact-container-name
value: master
- name: remote-moebius-url
value: http://0.0.0.0:31172/core/v3/application/definitions/pipeline/version
- name: sonar-host-url
value: http://sonar.xxx.com
- name: sonar-project-key
value: xxx-xxx
- name: dockerfile_arm
value: ""
- name: sonar-org
value: chameleon
- name: branch-name
value: main
- name: object-kind
value: push
- name: repo-url
value: git@xxx.git
- name: moebius-app-id
value: xxx
- name: sonar-project-version
value: main
- name: minio-access-key
value: xxx
- name: sonar-token
value: xxx
- name: minio-url
value: http://0.0.0.0:9000
- name: image-reference
value: docker.xxx.com/library/chameleon_demo
- name: extra-args
value: []
- name: moebius-app-version-id
value: 63bcdc7bab676200011918e1
pipelineRef:
name: go
podTemplate:
securityContext:
fsGroup: 65532
serviceAccountName: default
timeout: 1h0m0s
workspaces:
- emptyDir: {}
name: shared-data
- name: git-credentials
secret:
secretName: git-ssh-credential
- name: docker-config
secret:
secretName: docker-credential
status:
completionTime: "2023-01-19T04:12:04Z"
conditions:
- lastTransitionTime: "2023-01-19T04:12:04Z"
message: 'Tasks Completed: 3 (Failed: 0, Cancelled 0), Skipped: 3'
reason: Completed
status: "True"
type: Succeeded
pipelineSpec:
params:
- name: repo-url
type: string
- name: branch-name
type: string
- name: minio-url
type: string
- name: minio-access-key
type: string
- name: minio-secret-key
type: string
- name: sonar-host-url
type: string
- name: sonar-project-key
type: string
- name: sonar-project-version
type: string
- name: sonar-source-to-scan
type: string
- name: sonar-token
type: string
- name: sonar-org
type: string
- name: sonar-project-name
type: string
- name: image-reference
type: string
- name: extra-args
type: array
- name: dockerfile
type: string
- name: dockerfile_arm
type: string
- name: cpu-arch
type: array
- name: moebius-app-id
type: string
- name: moebius-app-version-id
type: string
- name: remote-moebius-url
type: string
- name: artifact-container-name
type: string
tasks:
- name: fetch-code
params:
- name: url
value: git@gitlab.xxx.com:xxx/chameleon_demo.git
- name: revision
value: main
- name: minio-url
value: http://0.0.0.0:9000
- name: minio-access-key
value: xxx
- name: minio-secret-key
value: xxx
taskRef:
kind: Task
name: git-cli
workspaces:
- name: output
workspace: shared-data
- name: ssh-directory
workspace: git-credentials
- name: scan-code
params:
- name: SONAR_HOST_URL
value: http://sonar.xxx.com
- name: SONAR_PROJECT_KEY
value: chameleon-xxx
- name: PROJECT_VERSION
value: main
- name: SOURCE_TO_SCAN
value: .
- name: SONAR_TOKEN
value: xxx
- name: SONAR_EXCLUSIONS
value: '**/*_test.go'
- name: SONAR_ORGANIZATION
value: chameleon
- name: SONAR_PROJECT_NAME
value: git@xxx.git
- name: minio-url
value: http://0.0.0.0:9000
- name: minio-access-key
value: xxx
- name: minio-secret-key
value: xxx
- name: s3-file
value: $(tasks.fetch-code.results.s3-file)
runAfter:
- fetch-code
taskRef:
kind: Task
name: sonarqube-scanner
workspaces:
- name: source
workspace: shared-data
- name: bake-image
params:
- name: IMAGE
value: docker.xxx.com/library/chameleon_demo
- name: EXTRA_ARGS
value: []
- name: DOCKERFILE
value: ./Dockerfile
- name: minio-url
value: http://0.0.0.0:9000
- name: minio-access-key
value: xxx
- name: minio-secret-key
value: xxx
- name: s3-file
value: $(tasks.fetch-code.results.s3-file)
- name: git-commit-id
value: $(tasks.fetch-code.results.commit)
- name: git-tag
value: $(tasks.fetch-code.results.tags)
runAfter:
- scan-code
taskRef:
kind: Task
name: kaniko
workspaces:
- name: source
workspace: shared-data
- name: dockerconfig
workspace: docker-config
- name: bake-image-arm
params:
- name: IMAGE
value: docker.xxx.com/library/chameleon_demo
- name: EXTRA_ARGS
value: []
- name: DOCKERFILE
value: ""
- name: minio-url
value: http://0.0.0.0:9000
- name: minio-access-key
value: xxx
- name: minio-secret-key
value: xxx
- name: s3-file
value: $(tasks.fetch-code.results.s3-file)
- name: git-commit-id
value: $(tasks.fetch-code.results.commit)
- name: git-tag
value: $(tasks.fetch-code.results.tags)
runAfter:
- scan-code
taskRef:
kind: Task
name: buildx
when:
- input: linux/arm64
operator: in
values:
- linux/amd64
workspaces:
- name: source
workspace: shared-data
- name: dockerconfig
workspace: docker-config
- name: publish-artifact-amd64
params:
- name: moebius-app-id
value: xxx
- name: moebius-app-version-id
value: xxx
- name: remote-moebius-url
value: http://0.0.0.0:31172/core/v3/application/definitions/pipeline/version
- name: artifact-container-name
value: master
- name: artifact-docker-image
value: $(tasks.bake-image.results.IMAGE_URL)
runAfter:
- bake-image
taskRef:
kind: Task
name: moebius-artifact-publish
when:
- input: $(tasks.fetch-code.results.need-publish-artifact)
operator: in
values:
- "yes"
- name: publish-artifact-arm64
params:
- name: moebius-app-id
value: xxx
- name: moebius-app-version-id
value: xxx
- name: remote-moebius-url
value: http://0.0.0.0:31172/core/v3/application/definitions/pipeline/version
- name: artifact-container-name
value: master
- name: artifact-docker-image
value: $(tasks.bake-image-arm.results.IMAGE_URL)
runAfter:
- bake-image-arm
taskRef:
kind: Task
name: moebius-artifact-publish
when:
- input: $(tasks.fetch-code.results.need-publish-artifact)
operator: in
values:
- "yes"
workspaces:
- name: shared-data
- name: git-credentials
- name: docker-config
skippedTasks:
- name: bake-image-arm
reason: When Expressions evaluated to false
whenExpressions:
- input: linux/arm64
operator: in
values:
- linux/amd64
- name: publish-artifact-amd64
reason: When Expressions evaluated to false
whenExpressions:
# NOW, you can see it, the INPUT value could not be replaced as well.
- input: $(tasks.fetch-code.results.need-publish-artifact)
operator: in
values:
- "yes"
- name: publish-artifact-arm64
reason: Results were missing
whenExpressions:
# NOW, you can see it, the INPUT value could not be replaced as well.
- input: $(tasks.fetch-code.results.need-publish-artifact)
operator: in
values:
- "yes"
startTime: "2023-01-19T04:09:40Z"
taskRuns:
63bcdc7bab676200011918e1-baac20fd-973e4a51-bake-image:
pipelineTaskName: bake-image
status:
completionTime: "2023-01-19T04:12:04Z"
conditions:
- lastTransitionTime: "2023-01-19T04:12:04Z"
message: All Steps have completed executing
reason: Succeeded
status: "True"
type: Succeeded
podName: 63bcdc7bab676200011918e1-baac20fd-973e4a51-bake-image-pod
startTime: "2023-01-19T04:10:16Z"
steps:
- container: step-uncache
imageID: docker-pullable://docker.xxx.com/global-tekton/minio-client@sha256:3e8796d9a496f8274d43055737bb7323a59e02e0c82d325c01b85194eceee6a6
name: uncache
terminated:
containerID: docker://edd8fd00ddeade1d151fdbb6086b500c28c7840ded8c163b2ff2d6cdee41bffc
exitCode: 0
finishedAt: "2023-01-19T04:10:01Z"
reason: Completed
startedAt: "2023-01-19T04:10:00Z"
- container: step-build-and-push
imageID: docker-pullable://docker.xxx.com/global-tekton/kaniko-project/executor@sha256:77cdc4f3262f0216a310dfd00ac5b7b0b0afe9466e662b0b439d13d11f41da5c
name: build-and-push
terminated:
containerID: docker://827bc051f5bb12540a1911a73121c252514654948f9461810f6e6d4f3b1d7445
exitCode: 0
finishedAt: "2023-01-19T04:11:41Z"
reason: Completed
startedAt: "2023-01-19T04:10:01Z"
taskSpec:
description: This Task builds a simple Dockerfile with kaniko and pushes
to a registry. This Task stores the image name and digest as results,
allowing Tekton Chains to pick up that an image was built & sign it.
params:
- description: Name (reference) of the image to build.
name: IMAGE
type: string
- default: ./Dockerfile
description: Path to the Dockerfile to build.
name: DOCKERFILE
type: string
- default: ./
description: The build context used by Kaniko.
name: CONTEXT
type: string
- name: EXTRA_ARGS
type: array
- default: docker.xxx.com/global-tekton/kaniko-project/executor:v1.5.1-debug
description: The image on which builds will run (default is v1.5.1)
name: BUILDER_IMAGE
type: string
- description: The remote minio url.
name: minio-url
type: string
- description: The remote minio url.
name: minio-access-key
type: string
- description: The remote minio url.
name: minio-secret-key
type: string
- description: The remote s3 file what fetched code saves to.
name: s3-file
type: string
- description: used for concating docker image tag.
name: git-commit-id
type: string
- description: used for concating docker image tag.
name: git-tag
type: string
results:
- description: Digest of the image just built.
name: IMAGE_DIGEST
type: string
- description: URL of the image just built.
name: IMAGE_URL
type: string
steps:
- image: docker.xxx.com/global-tekton/minio-client:2022.9.16-debian-11-r2
name: uncache
resources: {}
script: |
set -x
pwd
echo "remote s3 file: c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz"
file=`echo c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz | cut -f3 -d'/'`
echo "preparing download to local file: $file"
mc config host add c3 http://0.0.0.0:9000 xxx xxx
echo "checking if cache exists"
if mc cp c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz $file; then
tar -xzf $file -C /workspace/source/
ls /workspace/source
fi
- args:
- --dockerfile=./Dockerfile
- --context=/workspace/source/./
- --destination=docker.xxx.com/library/chameleon_demo:20230119-1209-baac20fd72f416c1cc41328a37f08a520c01255d-amd64
image: docker.xxx.com/global-tekton/kaniko-project/executor:v1.5.1-debug
name: build-and-push
resources: {}
securityContext:
runAsUser: 0
workingDir: /workspace/source
workspaces:
- description: Holds the context and Dockerfile
name: source
- description: Includes a docker `config.json`
mountPath: /kaniko/.docker
name: dockerconfig
optional: true
63bcdc7bab676200011918e1-baac20fd-973e4a51-fetch-code:
pipelineTaskName: fetch-code
status:
completionTime: "2023-01-19T04:10:03Z"
conditions:
- lastTransitionTime: "2023-01-19T04:10:03Z"
message: All Steps have completed executing
reason: Succeeded
status: "True"
type: Succeeded
podName: 63bcdc7bab676200011918e1-baac20fd-973e4a51-fetch-code-pod
startTime: "2023-01-19T04:09:56Z"
steps:
- container: step-clone
imageID: docker-pullable://docker.xxx.com/global-tekton/git@sha256:4f1352c11b90d1812b802036bdd03ec31833f07eb8f9bb2888e1fe69f0b8b3d5
name: clone
terminated:
containerID: docker://8697f1c833ee2a5a3ece4cd809b2147e47b12ac8b783f9f13cb59672f57b0d69
exitCode: 0
finishedAt: "2023-01-19T04:09:40Z"
message: '[{"key":"artifact-name","value":"20230119-1209","type":1},{"key":"branch","value":"main","type":1},{"key":"commit","value":"baac20fd72f416c1cc41328a37f08a520c01255d","type":1},{"key":"need-publish-artifact","value":"yes","type":1},{"key":"tags","value":"20230119-1209","type":1},{"key":"url","value":"git@gitlab.xxx.com:xxx/chameleon_demo.git","type":1}]'
reason: Completed
startedAt: "2023-01-19T04:09:39Z"
- container: step-cache
imageID: docker-pullable://docker.xxx.com/global-tekton/minio-client@sha256:3e8796d9a496f8274d43055737bb7323a59e02e0c82d325c01b85194eceee6a6
name: cache
terminated:
containerID: docker://2962b5ba0fe4b228f1454460053f2fca9caea271bd49c8e46d56ee593aea49e4
exitCode: 0
finishedAt: "2023-01-19T04:09:40Z"
message: '[{"key":"artifact-name","value":"20230119-1209","type":1},{"key":"branch","value":"main","type":1},{"key":"commit","value":"baac20fd72f416c1cc41328a37f08a520c01255d","type":1},{"key":"need-publish-artifact","value":"yes","type":1},{"key":"s3-file","value":"c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz","type":1},{"key":"tags","value":"20230119-1209","type":1},{"key":"url","value":"git@gitlab.xxx.com:xxx/chameleon_demo.git","type":1}]'
reason: Completed
startedAt: "2023-01-19T04:09:40Z"
taskResults:
- name: artifact-name
type: string
value: 20230119-1209
- name: branch
type: string
value: main
- name: commit
type: string
value: baac20fd72f416c1cc41328a37f08a520c01255d
- name: need-publish-artifact
type: string
value: "yes"
- name: tags
type: string
value: 20230119-1209
- name: url
type: string
value: git@gitlab.xxx.com:xxx/chameleon_demo.git
- name: s3-file
type: string
value: c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz
taskSpec:
description: |-
This task can be used to perform git operations.
Git command that needs to be run can be passed as a script to the task. This task needs authentication to git in order to push after the git operation.
params:
- default: docker.xxx.com/global-tekton/git:v2.26.2-reg
description: |
The base image for the task.
name: BASE_IMAGE
type: string
- default: ""
description: |
Git user name for performing git operation.
name: GIT_USER_NAME
type: string
- default: ""
description: |
Git user email for performing git operation.
name: GIT_USER_EMAIL
type: string
- description: Repository URL to clone from.
name: url
type: string
- default: ""
description: Revision to checkout. (branch, tag, sha, ref, etc...)
name: revision
type: string
- default: /root
description: |
Absolute path to the user's home directory. Set this explicitly if you are running the image as a non-root user or have overridden
the gitInitImage param with an image containing custom user configuration.
name: USER_HOME
type: string
- default: "true"
description: Log the commands that are executed during `git-clone`'s operation.
name: VERBOSE
type: string
- description: The remote minio url.
name: minio-url
type: string
- description: The remote minio url.
name: minio-access-key
type: string
- description: The remote minio url.
name: minio-secret-key
type: string
results:
- description: The precise commit SHA after the git operation.
name: commit
type: string
- description: The precise URL that was fetched by this Task.
name: url
type: string
- description: The remote s3 file what fetched code saves to.
name: s3-file
type: string
- description: The remote git repository branch currently in-used.
name: branch
type: string
- description: The git tags currently being tagged on the commit.
name: tags
type: string
- description: a flag indicated that current commit whether or not want
to publish as an artifact.
name: need-publish-artifact
type: string
- description: a name pointed to that the artifact name you want to publish.
name: artifact-name
type: string
steps:
- env:
- name: HOME
value: /root
- name: PARAM_URL
value: git@gitlab.xxx.com:xxx/chameleon_demo.git
- name: PARAM_REVISION
value: main
- name: PARAM_VERBOSE
value: "true"
- name: PARAM_USER_HOME
value: /root
- name: WORKSPACE_OUTPUT_PATH
value: /workspace/output
- name: WORKSPACE_SSH_DIRECTORY_BOUND
value: "true"
- name: WORKSPACE_SSH_DIRECTORY_PATH
value: /workspace/ssh-directory
- name: WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND
value: "false"
- name: WORKSPACE_BASIC_AUTH_DIRECTORY_PATH
image: docker.xxx.com/global-tekton/git:v2.26.2-reg
name: clone
resources: {}
script: |
#!/usr/bin/env sh
set -x
if [ "${PARAM_VERBOSE}" = "true" ] ; then
set -x
fi
if [ "${WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND}" = "true" ] ; then
cp "${WORKSPACE_BASIC_AUTH_DIRECTORY_PATH}/.git-credentials" "${PARAM_USER_HOME}/.git-credentials"
cp "${WORKSPACE_BASIC_AUTH_DIRECTORY_PATH}/.gitconfig" "${PARAM_USER_HOME}/.gitconfig"
chmod 400 "${PARAM_USER_HOME}/.git-credentials"
chmod 400 "${PARAM_USER_HOME}/.gitconfig"
fi
if [ "${WORKSPACE_SSH_DIRECTORY_BOUND}" = "true" ] ; then
cp -R "${WORKSPACE_SSH_DIRECTORY_PATH}" "${PARAM_USER_HOME}"/.ssh
chmod 700 "${PARAM_USER_HOME}"/.ssh
chmod -R 400 "${PARAM_USER_HOME}"/.ssh/*
fi
CHECKOUT_DIR="${WORKSPACE_OUTPUT_PATH}"
# clone到特定目录下
git clone -b "${PARAM_REVISION}" "${PARAM_URL}" "${CHECKOUT_DIR}"
cd "${CHECKOUT_DIR}"
ls -sla
RESULT_SHA="$(git rev-parse HEAD)"
EXIT_CODE="$?"
if [ "$EXIT_CODE" != 0 ]
then
exit $EXIT_CODE
fi
# Make sure we don't add a trailing newline to the result!
printf "%s" "$RESULT_SHA" > "/tekton/results/commit"
printf "%s" "${PARAM_URL}" > "/tekton/results/url"
printf "%s" "${PARAM_REVISION}" > "/tekton/results/branch"
TAGS=$(git tag --points-at HEAD)
if [[ -z $TAGS ]]
then
TAGS="dev"
fi
reg="(?<=(^|\s+)pub/)\S+(?=\s+|$)"
result=$(/bin/regh $TAGS $reg 0)
exitC=$?
resultYes="yes"
resultNo="no"
if [[ $exitC -eq "0" ]]
then
printf "%s" "${result}" > "/tekton/results/artifact-name"
printf "%s" "${result}" > "/tekton/results/tags"
printf "%s" "${resultYes}" > "/tekton/results/need-publish-artifact"
else
printf "%s" "" > "/tekton/results/artifact-name"
printf "%s" "${resultNo}" > "/tekton/results/need-publish-artifact"
if [[ -z $TAGS ]]
then
printf "%s" "dev" > "/tekton/results/tags"
else
printf "%s" "$TAGS" > "/tekton/results/tags"
fi
fi
ls -sla /tekton/results
workingDir: /workspace/output
- image: docker.xxx.com/global-tekton/minio-client:2022.9.16-debian-11-r2
name: cache
resources: {}
script: "commit=`cat /tekton/results/commit`\necho \"git commit id is
$commit\"\nmc config host add c3 http://0.0.0.0:9000 xxx
xxx\n\nfile=$commit.tar.gz\n\nif
mc stat c3/global-ci/$file; then\n echo \"cache already exists\"\nelse\n
\ cd /workspace/output \n tar -czf $file * \n mc cp $file c3/global-ci/$file\nfi\nprintf
\"%s\" \"c3/global-ci/$file\" > \"/tekton/results/s3-file\"\n"
workspaces:
- description: The git repo will be cloned onto the volume backing this
Workspace.
name: output
- description: |
An optional workspace that contains the files that need to be added to git. You can
access the workspace from your script using `$(workspaces.input.path)`, for instance:
cp $(workspaces.input.path)/file_that_i_want .
git add file_that_i_want
# etc
name: input
optional: true
- description: |
A .ssh directory with private key, known_hosts, config, etc. Copied to
the user's home before git commands are executed. Used to authenticate
with the git remote when performing the clone. Binding a Secret to this
Workspace is strongly recommended over other volume types.
name: ssh-directory
optional: true
- description: |
A Workspace containing a .gitconfig and .git-credentials file. These
will be copied to the user's home before any git commands are run. Any
other files in this Workspace are ignored. It is strongly recommended
to use ssh-directory over basic-auth whenever possible and to bind a
Secret to this Workspace over other volume types.
name: basic-auth
optional: true
63bcdc7bab676200011918e1-baac20fd-973e4a51-scan-code:
pipelineTaskName: scan-code
status:
completionTime: "2023-01-19T04:10:16Z"
conditions:
- lastTransitionTime: "2023-01-19T04:10:16Z"
message: All Steps have completed executing
reason: Succeeded
status: "True"
type: Succeeded
podName: 63bcdc7bab676200011918e1-baac20fd-973e4a51-scan-code-pod
startTime: "2023-01-19T04:10:03Z"
steps:
- container: step-uncache
imageID: docker-pullable://docker.xxx.com/global-tekton/minio-client@sha256:3e8796d9a496f8274d43055737bb7323a59e02e0c82d325c01b85194eceee6a6
name: uncache
terminated:
containerID: docker://41cb9950dfe8055d21c8fa69efa1d37a6fcde3d993e8892fd6bef2e0ef689413
exitCode: 0
finishedAt: "2023-01-19T04:09:46Z"
reason: Completed
startedAt: "2023-01-19T04:09:46Z"
- container: step-sonar-properties-create
imageID: docker-pullable://docker.xxx.com/global-tekton/ubi-minimal@sha256:f19c5b5d417cad1452ced0d174bca363ac41554190406c9147488b58394e2c56
name: sonar-properties-create
terminated:
containerID: docker://12f8ee936bc5dc6317f14f694e8111c47bb839a41faf03eb2b24736dd68a1d37
exitCode: 0
finishedAt: "2023-01-19T04:09:46Z"
message: '[{"key":"sonar_project_key","value":"chameleon-aec656c95f1e4201a993bb0522700e25","type":1}]'
reason: Completed
startedAt: "2023-01-19T04:09:46Z"
- container: step-sonar-scan
imageID: docker-pullable://docker.xxx.com/global-tekton/sonar-scanner-cli@sha256:edf5fd3f015f8a58e2b6f165a19b24cce66f9a767cf688ad9833f0104d87ff1f
name: sonar-scan
terminated:
containerID: docker://b6a61e9c1114fb961dd5cc4d41b6e1da9d034de1d1e6fa03cfa4d8aab4a29dd7
exitCode: 0
finishedAt: "2023-01-19T04:09:53Z"
message: '[{"key":"sonar_project_key","value":"chameleon-aec656c95f1e4201a993bb0522700e25","type":1}]'
reason: Completed
startedAt: "2023-01-19T04:09:46Z"
- container: step-collect-result
imageID: docker-pullable://docker.xxx.com/global-tekton/sonar-scanner-cli@sha256:edf5fd3f015f8a58e2b6f165a19b24cce66f9a767cf688ad9833f0104d87ff1f
name: collect-result
terminated:
containerID: docker://cb475e04922891c1d48f9937a891cb425e4753f56a9586735e5b99c8426d492f
exitCode: 0
finishedAt: "2023-01-19T04:09:54Z"
message: '[{"key":"sonar_project_key","value":"chameleon-aec656c95f1e4201a993bb0522700e25","type":1}]'
reason: Completed
startedAt: "2023-01-19T04:09:54Z"
taskResults:
- name: sonar_project_key
type: string
value: chameleon-aec656c95f1e4201a993bb0522700e25
taskSpec:
description: |-
The following task can be used to perform static analysis on the source code provided the SonarQube server is hosted
SonarQube is the leading tool for continuously inspecting the Code Quality and Security of your codebases, all while empowering development teams. Analyze over 25 popular programming languages including C#, VB.Net, JavaScript, TypeScript and C++. It detects bugs, vulnerabilities and code smells across project branches and pull requests.
params:
- default: ""
description: SonarQube server URL
name: SONAR_HOST_URL
type: string
- default: ""
description: Project's unique key
name: SONAR_PROJECT_KEY
type: string
- default: "1.0"
description: 'Version of the project. Default: 1.0'
name: PROJECT_VERSION
type: string
- default: .
description: Comma-separated paths to directories containing main source
files
name: SOURCE_TO_SCAN
type: string
- default: ""
description: The organization in sonarqube where the project exists
name: SONAR_ORGANIZATION
type: string
- default: docker.xxx.com/global-tekton/sonar-scanner-cli:4.6-jq
description: The sonarqube scanner CLI image which will run the scan
name: SONAR_SCANNER_IMAGE
type: string
- default: ""
description: Used to authenticate remote SonarQube
name: SONAR_TOKEN
type: string
- default: ""
description: Used to set exclusions to sonar.properties
name: SONAR_EXCLUSIONS
type: string
- default: ""
description: Used to set project name to sonar.properties
name: SONAR_PROJECT_NAME
type: string
- description: The remote minio url.
name: minio-url
type: string
- description: The remote minio url.
name: minio-access-key
type: string
- description: The remote minio url.
name: minio-secret-key
type: string
- description: The remote s3 file what fetched code saves to.
name: s3-file
type: string
results:
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_code_smells
type: string
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_bugs
type: string
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_vulnerabilities
type: string
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_sqale_index
type: string
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_code_smells
type: string
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_ncloc
type: string
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_coverage
type: string
- description: the result value being calculated from remote SonarQube based
on all of code commits.
name: total_duplicated_lines_density
type: string
- description: a project key that SonarQube used for calculating final dashboard
url.
name: sonar_project_key
type: string
steps:
- image: docker.xxx.com/global-tekton/minio-client:2022.9.16-debian-11-r2
name: uncache
resources: {}
script: |
pwd
echo "remote s3 file: c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz"
file=`echo c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz | cut -f3 -d'/'`
echo "preparing download to local file: $file"
mc config host add c3 http://0.0.0.0:9000 xxx xxx
echo "checking if cache exists"
if mc cp c3/global-ci/baac20fd72f416c1cc41328a37f08a520c01255d.tar.gz $file; then
tar -xzf $file -C /workspace/source/
ls /workspace/source
fi
- env:
- name: SONAR_HOST_URL
value: http://sonar.xxx.com
- name: SONAR_PROJECT_KEY
value: chameleon-aec656c95f1e4201a993bb0522700e25
- name: PROJECT_VERSION
value: main
- name: SOURCE_TO_SCAN
value: .
- name: SONAR_ORGANIZATION
value: chameleon
- name: SONAR_TOKEN
value: c72c6342c3676826a6dd42c2297aba2d11b312d1
- name: SONAR_EXCLUSIONS
value: '**/*_test.go'
- name: SONAR_PROJECT_NAME
value: git@gitlab.xxx.com:xxx/chameleon_demo.git
image: docker.xxx.com/global-tekton/ubi-minimal:8.2
name: sonar-properties-create
resources: {}
script: |
#!/usr/bin/env bash
replaceValues() {
filename=$1
thekey=$2
newvalue=$3
if ! grep -R "^[#]*\s*${thekey}=.*" $filename >/dev/null; then
echo "APPENDING because '${thekey}' not found"
echo "" >>$filename
echo "$thekey=$newvalue" >>$filename
else
echo "SETTING because '${thekey}' found already"
sed -ir "s|^[#]*\s*${thekey}=.*|$thekey=$newvalue|" $filename
fi
}
if [[ "false" == "true" ]]; then
if [[ -f /sonar-project.properties ]]; then
echo "using user provided sonar-project.properties file"
cp -RL /sonar-project.properties /workspace/source/sonar-project.properties
fi
fi
if [[ -f /workspace/source/sonar-project.properties ]]; then
if [[ -n "${SONAR_HOST_URL}" ]]; then
echo "replacing sonar host URL"
replaceValues /workspace/source/sonar-project.properties sonar.host.url "${SONAR_HOST_URL}"
fi
if [[ -n "${SONAR_PROJECT_KEY}" ]]; then
echo "replacing sonar project key"
replaceValues /workspace/source/sonar-project.properties sonar.projectKey "${SONAR_PROJECT_KEY}"
fi
echo "Values in sonar-project.properties file replaced successfully..."
else
echo "Creating sonar-project.properties file..."
touch sonar-project.properties
[[ -n "${SONAR_PROJECT_KEY}" ]] && {
echo "sonar.projectKey=${SONAR_PROJECT_KEY}" >> sonar-project.properties
} || {
echo "missing property SONAR_PROJECT_KEY"
exit 1
}
[[ -n "${SONAR_HOST_URL}" ]] && {
echo "sonar.host.url=${SONAR_HOST_URL}" >> sonar-project.properties
} || {
echo "missing property SONAR_HOST_URL"
exit 1
}
[[ -n "${PROJECT_VERSION}" ]] && {
echo "sonar.projectVersion=${PROJECT_VERSION}" >> sonar-project.properties
} || {
echo "missing property PROJECT_VERSION"
exit 1
}
[[ -n "${SONAR_ORGANIZATION}" ]] && {
echo "sonar.organization=${SONAR_ORGANIZATION}" >> sonar-project.properties
} || {
echo "missing property SONAR_ORGANIZATION"
exit 1
}
[[ -n "${SONAR_TOKEN}" ]] && {
echo "sonar.login=${SONAR_TOKEN}" >> sonar-project.properties
} || {
echo "missing property SONAR_TOKEN"
exit 1
}
[[ -n "${SONAR_PROJECT_NAME}" ]] && {
echo "sonar.projectName=${SONAR_PROJECT_NAME}" >> sonar-project.properties
} || {
echo "missing property SONAR_PROJECT_NAME"
exit 1
}
echo "sonar.exclusions=${SONAR_EXCLUSIONS}" >> sonar-project.properties
echo "sonar.sources=${SOURCE_TO_SCAN}" >> sonar-project.properties
echo "sonar.java.binaries=." >> sonar-project.properties
echo "sonar.java.source=1.8" >> sonar-project.properties
echo "---------------------------"
cat /workspace/source/sonar-project.properties
fi
printf "%s" "chameleon-aec656c95f1e4201a993bb0522700e25" > "/tekton/results/sonar_project_key"
workingDir: /workspace/source
- command:
- sonar-scanner
image: docker.xxx.com/global-tekton/sonar-scanner-cli:4.6-jq
name: sonar-scan
resources: {}
workingDir: /workspace/source
- image: docker.xxx.com/global-tekton/sonar-scanner-cli:4.6-jq
name: collect-result
resources: {}
script: |
#!/usr/bin/env bash
set -eu
cd "/workspace/source"
curl -u "c72c6342c3676826a6dd42c2297aba2d11b312d1:" -o sonar.result http://sonar.xxx.com/api/measures/component_tree\?metricKeys\=sqale_index,duplicated_lines_density,ncloc,coverage,bugs,code_smells,vulnerabilities\&component\=chameleon-aec656c95f1e4201a993bb0522700e25
for ((i=0;i<`cat sonar.result| jq -r ".baseComponent.measures | length"`;i++))
do
jq -r ".baseComponent.measures[$i]" sonar.result
printf "%.2f" $(jq -r ".baseComponent.measures[$i].value" sonar.result) > /tekton/results/total_$(jq -r ".baseComponent.measures[$i].metric" sonar.result)
done
ls -sla /tekton/results
workingDir: /workspace/source
workspaces:
- description: Workspace containing the code which needs to be scanned by
SonarQube
name: source
- description: Optional workspace where SonarQube properties can be mounted
name: sonar-settings
optional: true
It's a bit difficult to reproduce your issue, I tried a pipeline similar to yours, but didn't reproduce the problem, here it's my yaml:
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: start
namespace: default
spec:
results:
- description: /tmp/outputs/Output/data
name: need-publish-artifact
type: string
steps:
- command:
- sh
- -ec
- echo "yes" > $(results.need-publish-artifact.path)
image: python:alpine3.6
name: main
resources: {}
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: print-msg
namespace: default
spec:
steps:
- command:
- sh
- -ec
- echo "print-msg"
image: python:3.7
name: main
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: base
namespace: default
spec:
tasks:
- name: start
taskRef:
name: start
- name: print-msg
taskRef:
name: start
runAfter:
- start
when:
- input: $(tasks.start.results.need-publish-artifact)
operator: in
values:
- "yes"
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: pipeline2
namespace: default
spec:
pipelineRef:
name: base
@jerop could you take a look?
@chengjoey @jerop
Any updates?
@g0194776 please provide your config map setting if it's possible, for example:
kubectl get cm/feature-flags -o json -n tekton-pipelines | jq .data
{
"await-sidecar-readiness": "true",
"disable-affinity-assistant": "false",
"disable-creds-init": "false",
"embedded-status": "full",
"enable-api-fields": "stable",
"enable-custom-tasks": "false",
"enable-tekton-oci-bundles": "false",
"require-git-ssh-secret-known-hosts": "false",
"running-in-environment-with-injected-sidecars": "true",
"send-cloudevents-for-runs": "false"
}
Also, can you try running your pipeline against the latest tekton pipelines controller?
The result is populated as expected based on your status:
- name: need-publish-artifact
type: string
value: "yes"
I agree the skippedTasks
must have the value replaced in when expressions but I am not able to reproduce the issue your are running into:
# NOW, you can see it, the INPUT value could not be replaced as well.
- input: $(tasks.fetch-code.results.need-publish-artifact)
operator: in
values:
- "yes"
Can you try replacing yes
with something else and see what happens?
@pritidesai Thanks for your reply.
My configmap data is:
kubectl get cm/feature-flags -o json -n tekton-pipelines | jq .data
{
"await-sidecar-readiness": "true",
"disable-affinity-assistant": "false",
"disable-creds-init": "false",
"enable-api-fields": "stable",
"enable-custom-tasks": "false",
"enable-tekton-oci-bundles": "false",
"require-git-ssh-secret-known-hosts": "false",
"running-in-environment-with-injected-sidecars": "true",
"send-cloudevents-for-runs": "false"
}
Can you try replacing yes with something else and see what happens?
I have tested the words like true
but it does not work either.
Yep, I think I have to try to upgrade my Tekton version to the latest and retry it again, That's awful.
@chengjoey @pritidesai
I've tested with Tekton pipeline v0.44.0, It works well.
Even through the status field's value in the PipelineRun record had not been replaced, the when
expression also works.
status:
childReferences:
- apiVersion: tekton.dev/v1beta1
kind: TaskRun
name: 63bcdc7bab676200011918e1-2bj2t-fetch-code
pipelineTaskName: fetch-code
- apiVersion: tekton.dev/v1beta1
kind: TaskRun
name: 63bcdc7bab676200011918e1-2bj2t-scan-code
pipelineTaskName: scan-code
- apiVersion: tekton.dev/v1beta1
kind: TaskRun
name: 63bcdc7bab676200011918e1-2bj2t-bake-image
pipelineTaskName: bake-image
- apiVersion: tekton.dev/v1beta1
kind: TaskRun
name: 63bcdc7bab676200011918e1-2bj2t-publish-artifact-amd64
pipelineTaskName: publish-artifact-amd64
whenExpressions:
- input: $(tasks.fetch-code.results.need-publish-artifact)
operator: in
values:
- "yes"
completionTime: "2023-02-01T02:53:56Z"
conditions:
- lastTransitionTime: "2023-02-01T02:53:56Z"
message: 'Tasks Completed: 4 (Failed: 1, Cancelled 0), Skipped: 2'
reason: Failed
status: "False"
type: Succeeded
Thanks for both of your help.
Unfortunately, I encountered the same problem on version 0.41, is there any good way to troubleshoot.
tekton version is:
The push-flag results have been correctly set in taskrun, but the git-commit task is still skipped in the pipelinerun spec ![Uploading image.png…]()
Expected Behavior
I referred input value (e.g.
$(tasks.fetch-code.results.need-publish-artifact)
) can be replaced as well before the target task is prepared to run.Actual Behavior
Not get replaced and the target task has been skipped.
Additional Info
Kubernetes version:
Output of
kubectl version
:Tekton Pipeline version:
Output of
tkn version
orkubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'
I have debugged the code by dlv. I found that the input value what I referred in the when condition seems like not get replaced as well when the target task is preparing to run.
My Tekton pipeline definition is:
The
fetch-code
taskrun result is: