Tekton pipeline installation failing while running gitops

[indirakalagara]

Tekton pipeline installation failing while running gitops, with the below error . This is an intermittent issue, due to which sometimes end-to-end testing is failing.

│ Error: local-exec provisioner error │ │ with module.argocd-bootstrap.module.openshift_cicd.module.pipelines.null_resource.tekton_operator_helm, │ on .terraform/modules/argocd-bootstrap.openshift_cicd.pipelines/main.tf line 87, in resource "null_resource" "tekton_operator_helm": │ 87: provisioner "local-exec" { │ │ Error running command │ '.terraform/modules/argocd-bootstrap.openshift_cicd.pipelines/scripts/deploy-helm.sh │ 'openshift-operators' 'tekton' │ '.terraform/modules/argocd-bootstrap.openshift_cicd.pipelines/chart/tekton'': │ exit status 1. Output: WARNING: Kubernetes configuration file is │ group-readable. This is insecure. Location: │ /w/automation-aws-infra-openshift/workspaces/current/200-openshift-gitops/.tmp/.kube/config │ WARNING: Kubernetes configuration file is world-readable. This is insecure. │ Location: │ /w/automation-aws-infra-openshift/workspaces/current/200-openshift-gitops/.tmp/.kube/config │ --- │ # Source: tekton/charts/tool-config/templates/secret.yaml │ apiVersion: v1 │ kind: Secret │ metadata: │ name: tekton-access │ namespace: openshift-operators │ labels: │ helm.sh/chart: tool-config-0.13.0 │ app: tekton │ release: "tekton" │ app.kubernetes.io/part-of: tekton │ app.kubernetes.io/component: "tools" │ group: "catalyst-tools" │ grouping: "garage-cloud-native-toolkit" │ app.kubernetes.io/name: tekton │ app.kubernetes.io/instance: "tekton" │ app.kubernetes.io/version: "1.0" │ app.kubernetes.io/managed-by: Helm │ annotations: │ description: Secret to hold the username and password for tekton so that other components can access it │ type: Opaque │ stringData: │ TEKTON_URL: "https:///k8s/all-namespaces/tekton.dev~v1alpha1~Pipeline" │ url: "https:///k8s/all-namespaces/tekton.dev~v1alpha1~Pipeline" │ --- │ # Source: tekton/charts/tool-config/templates/config-map.yaml │ apiVersion: v1 │ kind: ConfigMap │ metadata: │ name: tekton-config │ namespace: openshift-operators │ labels: │ console-link.cloud-native-toolkit.dev/enabled: "false" │ helm.sh/chart: tool-config-0.13.0 │ app: tekton │ release: "tekton" │ app.kubernetes.io/part-of: tekton │ app.kubernetes.io/component: "tools" │ group: "catalyst-tools" │ grouping: "garage-cloud-native-toolkit" │ app.kubernetes.io/name: tekton │ app.kubernetes.io/instance: "tekton" │ app.kubernetes.io/version: "1.0" │ app.kubernetes.io/managed-by: Helm │ annotations: │ description: Config map to hold the url for tekton in the environment so that other components can access it │ console-link.cloud-native-toolkit.dev/section: "Cloud-Native Toolkit" │ console-link.cloud-native-toolkit.dev/location: ApplicationMenu │ console-link.cloud-native-toolkit.dev/displayName: Tekton │ console-link.cloud-native-toolkit.dev/imageUrl: "https://dashboard-tools.apps.zc-qs-cluster.p4x6.p1.openshiftapps.com/tools/icon/tekton" │ data: │ url: "https:///k8s/all-namespaces/tekton.dev~v1alpha1~Pipeline" │ TEKTON_URL: "https:///k8s/all-namespaces/tekton.dev~v1alpha1~Pipeline" │ --- │ # Source: tekton/charts/tekton-operator/templates/subscription.yaml │ apiVersion: operators.coreos.com/v1alpha1 │ kind: Subscription │ metadata: │ name: openshift-pipelines-operator-rh │ namespace: openshift-operators │ labels: │ helm.sh/chart: tekton-operator-0.3.2 │ app.kubernetes.io/name: tekton │ app.kubernetes.io/instance: tekton │ app.kubernetes.io/version: "1.18.0" │ created-by: "tekton-unknvjktjs90byv5" │ app.kubernetes.io/managed-by: Helm │ spec: │ channel: stable │ installPlanApproval: Automatic │ name: openshift-pipelines-operator-rh │ source: redhat-operators │ sourceNamespace: openshift-marketplace │ --- │ # Source: tekton/charts/tekton-operator/templates/post-install-hook.yaml │ apiVersion: v1 │ kind: ServiceAccount │ metadata: │ name: tekton-webhook-test │ labels: │ helm.sh/chart: tekton-operator-0.3.2 │ app.kubernetes.io/name: tekton │ app.kubernetes.io/instance: tekton │ app.kubernetes.io/version: "1.18.0" │ created-by: "tekton-unknvjktjs90byv5" │ app.kubernetes.io/managed-by: Helm │ annotations: │ "helm.sh/hook": post-install,post-upgrade │ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded │ --- │ # Source: tekton/charts/tekton-operator/templates/post-install-hook.yaml │ apiVersion: batch/v1 │ kind: Job │ metadata: │ name: tekton-webhook-test │ labels: │ helm.sh/chart: tekton-operator-0.3.2 │ app.kubernetes.io/name: tekton │ app.kubernetes.io/instance: tekton │ app.kubernetes.io/version: "1.18.0" │ created-by: "tekton-unknvjktjs90byv5" │ app.kubernetes.io/managed-by: Helm │ annotations: │ "helm.sh/hook": post-install,post-upgrade │ "helm.sh/hook-delete-policy": before-hook-creation │ spec: │ ttlSecondsAfterFinished: 300 │ template: │ spec: │ serviceAccountName: tekton-webhook-test │ initContainers: │ - name: wait-for-tekton-webhook │ image: quay.io/ibmgaragecloud/alpine-curl │ imagePullPolicy: IfNotPresent │ env: │ - name: URL │ value: http://tekton-pipelines-webhook.openshift-pipelines.svc:8080 │ command: ["sh"] │ args: │ - "-c" │ - "count=0; until curl -Iskf ${URL} || [[ $count -eq 20 ]]; do echo \">>> waiting for ${URL}\"; sleep 90; count=$((count + 1)); done; if [[ $count -eq 20 ]]; then echo \"Timeout\"; exit 1; else echo \">>> Started\"; fi" │ containers: │ - name: tekton-webhook-started │ image: quay.io/ibmgaragecloud/alpine-curl │ imagePullPolicy: Always │ env: │ - name: URL │ value: http://tekton-pipelines-webhook.openshift-pipelines.svc:8080 │ command: ["sh"] │ args: │ - "-c" │ - "curl -Iskf ${URL}" │ restartPolicy: Never │ backoffLimit: 1 │ WARNING: Kubernetes configuration file is group-readable. This is insecure. │ Location: │ /w/automation-aws-infra-openshift/workspaces/current/200-openshift-gitops/.tmp/.kube/config │ WARNING: Kubernetes configuration file is world-readable. This is insecure. │ Location: │ /w/automation-aws-infra-openshift/workspaces/current/200-openshift-gitops/.tmp/.kube/config │ Release "tekton" does not exist. Installing it now. │ Error: failed post-install: timed out waiting for the condition │ ╵ time=2022-08-24T06:31:05Z level=info msg=Encountered an error eligible for retrying. Sleeping 1m0s before retrying. prefix=[/__w/automation-aws-infra-openshift/workspaces/current/200-openshift-gitops] null_resource.print_200_inputs: Refreshing state... [id=4940062377572178468]

Log location: https://github.com/IBM/automation-aws-infra-openshift/runs/7987943110?check_suite_focus=true#step:4:6144

[indirakalagara]

fyi @rich-ehrhardt

[timroster]

This can be an intermittent error depending on cluster performance when deploying the chart to install tekton. Helm update https://github.com/cloud-native-toolkit/terraform-tools-tekton/blob/044753ef4b2bf96b5e34a6c3b5f0ee6aeddef267/scripts/deploy-helm.sh#L45 has a default timeout of 5 minutes and if it takes longer than this for the post-install deployment of the webhook-test job, then this error will appear.