Failing to apply customresourcedefinitions at line Makefile.mk:291

[nishantn3]

I have deployed using a separate method. Though I have deployed eks, roles and other resources using terraform template but I am deploying the jx/jx-git-operator manually using helm since our cluster is completely private and terraform cloud is not able to communicate with it.

I have applied the terraform-jx-requirements configmap and jx/jx-git-operator helm chart manually. I have also the the variables required to run this chart.

The jx-git-operator pod is up and running but the jx-boot job is failing. At the end I see that kubectl apply is failing at line 291. Where the customresourcedefinitions are getting applied from config-root folder. So I copied the config-root folder from pod and tried running it from the host and it is failing with error: no objects passed to apply.

I am adding the logs below from the jx-boot pod.

Please let me know if you need any other logs.

Thanks.

jx gitops git setup
setup git user  email jenkins-x@googlegroups.com
generated Git credentials file: /workspace/xdg_config/git/credentials with username: jx-bot email:
jx gitops apply
found last commit message: Update README
make regen-phase-1
make[1]: Entering directory '/workspace/source'
jx gitops git setup
setup git user  email jenkins-x@googlegroups.com
generated Git credentials file: /workspace/xdg_config/git/credentials with username: jx-bot email:
Already up to date.
jx gitops requirements merge
wrote the ConfigMap jx-requirements.yml to /tmp/3386116303
saved file /workspace/source/jx-requirements.yml
jx gitops requirements resolve -n
modified the chart repository in /workspace/source/jx-requirements.yml
WARNING: no email in secret jx-boot in namespace default
modified the pipeline user in /workspace/source/jx-requirements.yml
no resolve logic for kubernetes provider eks
jx gitops repository create
file does not exist: .jx/gitops/source-config.yaml so not defaulting any SourceConfig resources
jx gitops repository resolve --source-dir config-root/namespaces
jx gitops jenkins jobs
the source config file .jx/gitops/source-config.yaml does not exist
jx secret convert edit
jx gitops helmfile resolve
jx gitops image -s .jx/git-operator
helmfile --file helmfile.yaml template --validate --include-crds --output-dir-template /tmp/generate/{{.Release.Namespace}}/{{.Release.Name}}
Adding repo jetstack https://charts.jetstack.io
"jetstack" has been added to your repositories

Adding repo bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

Adding repo stable https://charts.helm.sh/stable
"stable" has been added to your repositories

Adding repo jxgh https://jenkins-x-charts.github.io/repo
"jxgh" has been added to your repositories

Templating release=jxboot-helmfile-resources, chart=jxgh/jxboot-helmfile-resources
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/build-bot-serviceaccount.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcactivities-sa.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcjobs-sa.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcpods-sa.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-basic-auth-htpasswd-secret.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-basic-auth-user-password-secret.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/maven-settings-secret.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/tekton-container-registry-auth-secret.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/tekton-git-secret.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/ingress-config-configmap.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jenkins-x-docker-registry.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jenkins-x-extensions-config.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/kapp-config.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/build-bot-clusterrole.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jenkins-x-aggregate-view.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcjobs-cr.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/build-bot-clusterrolebinding.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcjobs-crb.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/build-bot-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/committer-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcactivities-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcpods-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-pipeline-activity-reporter-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-view-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/owner-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/viewer-role.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/build-bot-rolebinding.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcactivities-rb.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcpods-rb.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcactivities-cronjob.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcjobs-cronjob.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/jx-gcpods-cronjob.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/700-chartmuseum-ing.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/700-hook-ing.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/700-nexus-ing.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/environments.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/environments.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/environments.yaml
wrote /tmp/generate/jx/jxboot-helmfile-resources/jxboot-helmfile-resources/templates/repositories.yaml

Templating release=jenkins-x-crds, chart=jxgh/jenkins-x-crds
wrote /tmp/generate/jx/jenkins-x-crds/jenkins-x-crds/templates/environments-crd.yaml
wrote /tmp/generate/jx/jenkins-x-crds/jenkins-x-crds/templates/pipelineactivities-crd.yaml
wrote /tmp/generate/jx/jenkins-x-crds/jenkins-x-crds/templates/releases-crd.yaml
wrote /tmp/generate/jx/jenkins-x-crds/jenkins-x-crds/templates/sourcerepositories-crd.yaml

Templating release=jx-pipelines-visualizer, chart=jxgh/jx-pipelines-visualizer
wrote /tmp/generate/jx/jx-pipelines-visualizer/jx-pipelines-visualizer/templates/rbac.yaml
wrote /tmp/generate/jx/jx-pipelines-visualizer/jx-pipelines-visualizer/templates/rbac.yaml
wrote /tmp/generate/jx/jx-pipelines-visualizer/jx-pipelines-visualizer/templates/rbac.yaml
wrote /tmp/generate/jx/jx-pipelines-visualizer/jx-pipelines-visualizer/templates/service.yaml
wrote /tmp/generate/jx/jx-pipelines-visualizer/jx-pipelines-visualizer/templates/deployment.yaml
wrote /tmp/generate/jx/jx-pipelines-visualizer/jx-pipelines-visualizer/templates/ingress.yaml

Templating release=jx-preview, chart=jxgh/jx-preview
wrote /tmp/generate/jx/jx-preview/jx-preview/templates/gc-jobs-sa.yaml
wrote /tmp/generate/jx/jx-preview/jx-preview/templates/previews.preview.jenkins.io.yaml
wrote /tmp/generate/jx/jx-preview/jx-preview/templates/gc-jobs-clusterrole.yaml
wrote /tmp/generate/jx/jx-preview/jx-preview/templates/gc-jobs-clusterrolebinding.yaml
wrote /tmp/generate/jx/jx-preview/jx-preview/templates/gc-jobs-role.yaml
wrote /tmp/generate/jx/jx-preview/jx-preview/templates/gc-jobs-rb.yaml
wrote /tmp/generate/jx/jx-preview/jx-preview/templates/gc-jobs-cj.yaml

Templating release=lighthouse, chart=jxgh/lighthouse
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/foghorn-sa.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/gc-jobs-sa.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/keeper-sa.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/tekton-controller-sa.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/webhooks-sa.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/hmacsecret.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/oauthsecret.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/config-external-plugins.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/lighthousebreakpoints-crd.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/lighthousejobs-crd.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/foghorn-role.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/gc-jobs-role.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/keeper-role.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/tekton-controller-role.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/webhooks-role.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/foghorn-rb.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/gc-jobs-rb.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/keeper-rb.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/tekton-controller-rb.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/webhooks-rb.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/keeper-service.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/tekton-controller-service.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/webhooks-service.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/foghorn-deployment.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/keeper-deployment.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/tekton-controller-deployment.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/webhooks-deployment.yaml
wrote /tmp/generate/jx/lighthouse/lighthouse/templates/gc-jobs-cj.yaml

Templating release=nexus, chart=jxgh/nexus
wrote /tmp/generate/jx/nexus/nexus/templates/secret.yaml
wrote /tmp/generate/jx/nexus/nexus/templates/config.yaml
wrote /tmp/generate/jx/nexus/nexus/templates/pvc.yaml
wrote /tmp/generate/jx/nexus/nexus/templates/service.yaml
wrote /tmp/generate/jx/nexus/nexus/templates/deployment.yaml

Templating release=chartmuseum, chart=stable/chartmuseum
wrote /tmp/generate/jx/chartmuseum/chartmuseum/templates/secret.yaml
wrote /tmp/generate/jx/chartmuseum/chartmuseum/templates/pvc.yaml
wrote /tmp/generate/jx/chartmuseum/chartmuseum/templates/service.yaml
wrote /tmp/generate/jx/chartmuseum/chartmuseum/templates/deployment.yaml

Templating release=jx-build-controller, chart=jxgh/jx-build-controller
wrote /tmp/generate/jx/jx-build-controller/jx-build-controller/templates/serviceaccount.yaml
wrote /tmp/generate/jx/jx-build-controller/jx-build-controller/templates/clusterrole.yaml
wrote /tmp/generate/jx/jx-build-controller/jx-build-controller/templates/clusterrolebinding.yaml
wrote /tmp/generate/jx/jx-build-controller/jx-build-controller/templates/role.yaml
wrote /tmp/generate/jx/jx-build-controller/jx-build-controller/templates/rolebinding.yaml
wrote /tmp/generate/jx/jx-build-controller/jx-build-controller/templates/deployment.yaml

Templating release=health-checks-jx, chart=jxgh/jx-kh-check
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-check.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-events-check.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-check.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-events-check.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-check.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-events-check.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-bot-token.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-check.yaml
wrote /tmp/generate/jx/health-checks-jx/jx-kh-check/templates/jx-webhook-events-check.yaml

Adding repo ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories

Templating release=nginx-ingress, chart=ingress-nginx/ingress-nginx
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-poddisruptionbudget.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-serviceaccount.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-configmap.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/clusterrole.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/clusterrolebinding.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-role.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-rolebinding.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-service-metrics.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-service-webhook.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-service.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-deployment.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/controller-ingressclass.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
wrote /tmp/generate/nginx/nginx-ingress/ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml

Adding repo external-secrets https://external-secrets.github.io/kubernetes-external-secrets
"external-secrets" has been added to your repositories

Adding repo banzaicloud-stable https://kubernetes-charts.banzaicloud.com
"banzaicloud-stable" has been added to your repositories

Templating release=kubernetes-external-secrets, chart=external-secrets/kubernetes-external-secrets
wrote /tmp/generate/secret-infra/kubernetes-external-secrets/kubernetes-external-secrets/crds/kubernetes-client.io_externalsecrets_crd.yaml
wrote /tmp/generate/secret-infra/kubernetes-external-secrets/kubernetes-external-secrets/templates/serviceaccount.yaml
wrote /tmp/generate/secret-infra/kubernetes-external-secrets/kubernetes-external-secrets/templates/rbac.yaml
wrote /tmp/generate/secret-infra/kubernetes-external-secrets/kubernetes-external-secrets/templates/rbac.yaml
wrote /tmp/generate/secret-infra/kubernetes-external-secrets/kubernetes-external-secrets/templates/rbac.yaml
wrote /tmp/generate/secret-infra/kubernetes-external-secrets/kubernetes-external-secrets/templates/service.yaml
wrote /tmp/generate/secret-infra/kubernetes-external-secrets/kubernetes-external-secrets/templates/deployment.yaml

Templating release=pusher-wave, chart=jxgh/pusher-wave
wrote /tmp/generate/secret-infra/pusher-wave/pusher-wave/templates/serviceaccount.yaml
wrote /tmp/generate/secret-infra/pusher-wave/pusher-wave/templates/clusterrole.yaml
wrote /tmp/generate/secret-infra/pusher-wave/pusher-wave/templates/clusterrolebinding.yaml
wrote /tmp/generate/secret-infra/pusher-wave/pusher-wave/templates/deployment.yaml

Adding repo cdf https://cdfoundation.github.io/tekton-helm-chart
"cdf" has been added to your repositories

Templating release=tekton-pipeline, chart=cdf/tekton-pipeline
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-ns.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-podsecuritypolicy.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-bot-sa.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-sa.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-sa.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/webhook-certs-secret.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config-artifact-bucket-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config-artifact-pvc-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config-defaults-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config-leader-election-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config-logging-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config-observability-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config-registry-cert-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/feature-flags-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/pipelines-info-cm.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/clustertasks.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/conditions.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/pipelineresources.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/pipelineruns.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/pipelines.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/runs.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/taskruns.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tasks.tekton.dev-crd.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-aggregate-edit-clusterrole.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-aggregate-view-clusterrole.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-cluster-access-clusterrole.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-tenant-access-clusterrole.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-cluster-access-clusterrole.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-cluster-access-crb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-tenant-access-crb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-cluster-access-crb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-role.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-info-role.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-leader-election-role.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-role.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-leaderelection-rb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-rb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-info-rb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-leaderelection-rb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-rb.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-svc.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-svc.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-deploy.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-deploy.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-horizontalpodautoscaler.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/webhook.pipeline.tekton.dev-mutwebhookcfg.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config.webhook.pipeline.tekton.dev-valwebhookcfg.yaml
wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/validation.webhook.pipeline.tekton.dev-valwebhookcfg.yaml

jx gitops split --dir /tmp/generate
jx gitops rename --dir /tmp/generate
jx gitops helmfile move --output-dir config-root --dir /tmp/generate --dir-includes-release-name
jx secret convert --source-dir config-root -r jx-vault -m kubernetes
jx secret replicate --selector secret.jenkins-x.io/replica-source=true
jx gitops namespace --dir-mode --dir config-root/namespaces
jx gitops helmfile report
namespace cert-manager
ignoring release jetstack/cert-manager in namespace cert-manager as using conditional jxRequirementsIngressTLS.enabled

namespace cert-manager
ignoring release jetstack/cert-manager in namespace cert-manager as using conditional jxRequirementsIngressTLS.enabled

namespace jx
ignoring release bitnami/external-dns in namespace jx as using conditional jxRequirementsIngressExternalDNS.enabled
ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/chartmuseum-ingress.yaml with api version networking.k8s.io/v1
ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/hook-ingress.yaml with api version networking.k8s.io/v1
ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/nexus-ingress.yaml with api version networking.k8s.io/v1
found jxboot-helmfile-resources version: 1.1.112 https://github.com/jenkins-x-charts/jxboot-helmfile-resources
WARNING: ignoring release jxgh/jenkins-x-crds in namespace jx as we cannot find any generated resources but there is no conditional
ignoring Ingress in file config-root/namespaces/jx/jx-pipelines-visualizer/jx-pipelines-visualizer-ingress.yaml with api version networking.k8s.io/v1
found jx-pipelines-visualizer version: 1.8.2 https://github.com/jenkins-x/jx-pipelines-visualizer
found jx-preview version: 0.0.206 https://github.com/jenkins-x-plugins/jx-preview
found lighthouse version: 1.6.5 https://github.com/jenkins-x/lighthouse
found nexus version: 0.1.41 https://github.com/jenkins-x-charts/nexus
found chartmuseum version: 2.14.2 https://github.com/helm/chartmuseum
found jx-build-controller version: 0.3.21 https://jenkins-x.io/
ignoring release jxgh/acme in namespace jx as using conditional jxRequirementsIngressTLS.enabled
found jx-kh-check version: 0.0.78 https://jenkins-x.io/

namespace jx
ignoring release bitnami/external-dns in namespace jx as using conditional jxRequirementsIngressExternalDNS.enabled
ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/chartmuseum-ingress.yaml with api version networking.k8s.io/v1
ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/hook-ingress.yaml with api version networking.k8s.io/v1
ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/nexus-ingress.yaml with api version networking.k8s.io/v1
found jxboot-helmfile-resources version: 1.1.112 https://github.com/jenkins-x-charts/jxboot-helmfile-resources
WARNING: ignoring release jxgh/jenkins-x-crds in namespace jx as we cannot find any generated resources but there is no conditional
ignoring Ingress in file config-root/namespaces/jx/jx-pipelines-visualizer/jx-pipelines-visualizer-ingress.yaml with api version networking.k8s.io/v1
found jx-pipelines-visualizer version: 1.8.2 https://github.com/jenkins-x/jx-pipelines-visualizer
found jx-preview version: 0.0.206 https://github.com/jenkins-x-plugins/jx-preview
found lighthouse version: 1.6.5 https://github.com/jenkins-x/lighthouse
found nexus version: 0.1.41 https://github.com/jenkins-x-charts/nexus
found chartmuseum version: 2.14.2 https://github.com/helm/chartmuseum
found jx-build-controller version: 0.3.21 https://jenkins-x.io/
ignoring release jxgh/acme in namespace jx as using conditional jxRequirementsIngressTLS.enabled
found jx-kh-check version: 0.0.78 https://jenkins-x.io/

namespace kuberhealthy
ignoring release jxgh/kh-tls-check in namespace kuberhealthy as using conditional jxRequirementsTLSCheck.enabled

namespace kuberhealthy
ignoring release jxgh/kh-tls-check in namespace kuberhealthy as using conditional jxRequirementsTLSCheck.enabled

namespace nginx
found ingress-nginx version: 4.1.0 https://github.com/kubernetes/ingress-nginx

namespace nginx
found ingress-nginx version: 4.1.0 https://github.com/kubernetes/ingress-nginx

namespace secret-infra
found kubernetes-external-secrets version: 8.5.5 https://github.com/external-secrets/kubernetes-external-secrets
found pusher-wave version: 0.4.21 https://github.com/pusher/wave

namespace secret-infra
found kubernetes-external-secrets version: 8.5.5 https://github.com/external-secrets/kubernetes-external-secrets
found pusher-wave version: 0.4.21 https://github.com/pusher/wave

namespace tekton-pipelines
found tekton-pipeline version: 0.29.0 https://github.com/cdfoundation/tekton-helm-chart

namespace tekton-pipelines
found tekton-pipeline version: 0.29.0 https://github.com/cdfoundation/tekton-helm-chart

saved docs/releases.yaml
saved docs/README.md
jx gitops scheduler
jx gitops hash --pod-spec --kind Deployment -s config-root/namespaces/jx/lighthouse-config/config-cm.yaml -s config-root/namespaces/jx/lighthouse-config/plugins-cm.yaml -d config-root/namespaces/jx/lighthouse
jx gitops label --dir config-root/cluster                   gitops.jenkins-x.io/pipeline=cluster
jx gitops label --dir config-root/customresourcedefinitions gitops.jenkins-x.io/pipeline=customresourcedefinitions
jx gitops label --dir config-root/namespaces                gitops.jenkins-x.io/pipeline=namespaces
jx gitops annotate --dir config-root --selector app=pusher-wave kapp.k14s.io/change-group=apps.jenkins-x.io/pusher-wave
jx gitops annotate --dir config-root --selector app.kubernetes.io/name=ingress-nginx kapp.k14s.io/change-group=apps.jenkins-x.io/ingress-nginx
jx gitops label --dir config-root/cluster --kind=Namespace team=jx
jx gitops annotate --dir  config-root/namespaces --kind Deployment --selector app=pusher-wave --invert-selector wave.pusher.com/update-on-config-change=true
using kubectl to apply resources
kubectl apply --force --prune -l=gitops.jenkins-x.io/pipeline=customresourcedefinitions -R -f config-root/customresourcedefinitions
customresourcedefinition.apiextensions.k8s.io/environments.jenkins.io configured
customresourcedefinition.apiextensions.k8s.io/pipelineactivities.jenkins.io configured
customresourcedefinition.apiextensions.k8s.io/releases.jenkins.io configured
customresourcedefinition.apiextensions.k8s.io/sourcerepositories.jenkins.io configured
customresourcedefinition.apiextensions.k8s.io/previews.preview.jenkins.io unchanged
customresourcedefinition.apiextensions.k8s.io/lighthousebreakpoints.lighthouse.jenkins.io unchanged
customresourcedefinition.apiextensions.k8s.io/lighthousejobs.lighthouse.jenkins.io unchanged
customresourcedefinition.apiextensions.k8s.io/externalsecrets.kubernetes-client.io unchanged
customresourcedefinition.apiextensions.k8s.io/clustertasks.tekton.dev configured
customresourcedefinition.apiextensions.k8s.io/conditions.tekton.dev unchanged
customresourcedefinition.apiextensions.k8s.io/pipelineresources.tekton.dev unchanged
customresourcedefinition.apiextensions.k8s.io/pipelineruns.tekton.dev configured
customresourcedefinition.apiextensions.k8s.io/pipelines.tekton.dev configured
customresourcedefinition.apiextensions.k8s.io/runs.tekton.dev configured
customresourcedefinition.apiextensions.k8s.io/taskruns.tekton.dev configured
customresourcedefinition.apiextensions.k8s.io/tasks.tekton.dev configured
kubectl apply --force --prune -l=gitops.jenkins-x.io/pipeline=cluster                   -R -f config-root/cluster
namespace/jx-production configured
namespace/jx-staging configured
namespace/jx unchanged
namespace/nginx configured
namespace/secret-infra configured
clusterrole.rbac.authorization.k8s.io/jx-build-controller-jx unchanged
clusterrolebinding.rbac.authorization.k8s.io/jx-build-controller-jx unchanged
clusterrole.rbac.authorization.k8s.io/jx-pipelines-visualizer unchanged
clusterrolebinding.rbac.authorization.k8s.io/jx-pipelines-visualizer unchanged
clusterrole.rbac.authorization.k8s.io/jx-preview-gc-jobs unchanged
clusterrolebinding.rbac.authorization.k8s.io/jx-preview-gc-jobs unchanged
clusterrole.rbac.authorization.k8s.io/gcjobs unchanged
clusterrolebinding.rbac.authorization.k8s.io/gcjobs unchanged
clusterrole.rbac.authorization.k8s.io/jenkinsx-aggregate-view unchanged
clusterrole.rbac.authorization.k8s.io/tekton-bot unchanged
clusterrolebinding.rbac.authorization.k8s.io/tekton-bot-jx unchanged
clusterrole.rbac.authorization.k8s.io/ingress-nginx unchanged
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-external-secrets-auth unchanged
clusterrole.rbac.authorization.k8s.io/kubernetes-external-secrets unchanged
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-external-secrets unchanged
clusterrole.rbac.authorization.k8s.io/pusher-wave-pusher-wave unchanged
clusterrolebinding.rbac.authorization.k8s.io/pusher-wave-pusher-wave unchanged
clusterrole.rbac.authorization.k8s.io/tekton-aggregate-edit unchanged
clusterrole.rbac.authorization.k8s.io/tekton-aggregate-view unchanged
clusterrole.rbac.authorization.k8s.io/tekton-pipelines-controller-cluster-access unchanged
clusterrolebinding.rbac.authorization.k8s.io/tekton-pipelines-controller-cluster-access unchanged
clusterrole.rbac.authorization.k8s.io/tekton-pipelines-controller-tenant-access unchanged
clusterrolebinding.rbac.authorization.k8s.io/tekton-pipelines-controller-tenant-access unchanged
namespace/tekton-pipelines unchanged
clusterrole.rbac.authorization.k8s.io/tekton-pipelines-webhook-cluster-access unchanged
clusterrolebinding.rbac.authorization.k8s.io/tekton-pipelines-webhook-cluster-access unchanged
kubectl apply --force --prune -l=gitops.jenkins-x.io/pipeline=namespaces                -R -f config-root/namespaces
deployment.apps/jenkins-x-chartmuseum configured
persistentvolumeclaim/jenkins-x-chartmuseum unchanged
externalsecret.kubernetes-client.io/jenkins-x-chartmuseum unchanged
service/jenkins-x-chartmuseum unchanged
serviceaccount/jenkins-x-controllerbuild unchanged
deployment.apps/jx-build-controller configured
rolebinding.rbac.authorization.k8s.io/jx-build-controller unchanged
role.rbac.authorization.k8s.io/jx-build-controller unchanged
rolebinding.rbac.authorization.k8s.io/jx-webhook-check-rb unchanged
rolebinding.rbac.authorization.k8s.io/jx-webhook-events-check-rb unchanged
serviceaccount/jx-webhook-events-sa unchanged
role.rbac.authorization.k8s.io/jx-webhook-events-service-role unchanged
serviceaccount/jx-webhook-sa unchanged
role.rbac.authorization.k8s.io/jx-webhook-service-role unchanged
deployment.apps/jx-pipelines-visualizer configured
ingress.networking.k8s.io/jx-pipelines-visualizer unchanged
serviceaccount/jx-pipelines-visualizer unchanged
service/jx-pipelines-visualizer unchanged
Warning: batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob
cronjob.batch/jx-preview-gc-jobs unchanged
rolebinding.rbac.authorization.k8s.io/jx-preview-gc-jobs unchanged
role.rbac.authorization.k8s.io/jx-preview-gc-jobs unchanged
serviceaccount/jx-preview-gc-jobs unchanged
ingress.networking.k8s.io/chartmuseum unchanged
role.rbac.authorization.k8s.io/committer unchanged
environment.jenkins.io/dev unchanged
sourcerepository.jenkins.io/dev unchanged
rolebinding.rbac.authorization.k8s.io/gcactivities unchanged
role.rbac.authorization.k8s.io/gcactivities unchanged
rolebinding.rbac.authorization.k8s.io/gcpods unchanged
role.rbac.authorization.k8s.io/gcpods unchanged
ingress.networking.k8s.io/hook unchanged
configmap/ingress-config unchanged
externalsecret.kubernetes-client.io/jenkins-maven-settings unchanged
configmap/jenkins-x-docker-registry unchanged
configmap/jenkins-x-extensions unchanged
externalsecret.kubernetes-client.io/jx-basic-auth-htpasswd unchanged
externalsecret.kubernetes-client.io/jx-basic-auth-user-password unchanged
cronjob.batch/jx-gcactivities configured
serviceaccount/jx-gcactivities unchanged
cronjob.batch/jx-gcjobs configured
serviceaccount/jx-gcjobs unchanged
cronjob.batch/jx-gcpods configured
serviceaccount/jx-gcpods unchanged
role.rbac.authorization.k8s.io/jx-pipeline-activity-updater unchanged
role.rbac.authorization.k8s.io/jx-view unchanged
configmap/kapp-config unchanged
ingress.networking.k8s.io/nexus unchanged
role.rbac.authorization.k8s.io/owner unchanged
environment.jenkins.io/production unchanged
environment.jenkins.io/staging unchanged
rolebinding.rbac.authorization.k8s.io/tekton-bot unchanged
role.rbac.authorization.k8s.io/tekton-bot unchanged
serviceaccount/tekton-bot configured
externalsecret.kubernetes-client.io/tekton-container-registry-auth unchanged
externalsecret.kubernetes-client.io/tekton-git unchanged
role.rbac.authorization.k8s.io/viewer unchanged
service/hook unchanged
configmap/lighthouse-external-plugins unchanged
deployment.apps/lighthouse-foghorn configured
rolebinding.rbac.authorization.k8s.io/lighthouse-foghorn unchanged
role.rbac.authorization.k8s.io/lighthouse-foghorn unchanged
serviceaccount/lighthouse-foghorn unchanged
cronjob.batch/lighthouse-gc-jobs unchanged
rolebinding.rbac.authorization.k8s.io/lighthouse-gc-jobs unchanged
role.rbac.authorization.k8s.io/lighthouse-gc-jobs unchanged
serviceaccount/lighthouse-gc-jobs unchanged
externalsecret.kubernetes-client.io/lighthouse-hmac-token unchanged
deployment.apps/lighthouse-keeper configured
rolebinding.rbac.authorization.k8s.io/lighthouse-keeper unchanged
role.rbac.authorization.k8s.io/lighthouse-keeper unchanged
serviceaccount/lighthouse-keeper unchanged
service/lighthouse-keeper unchanged
externalsecret.kubernetes-client.io/lighthouse-oauth-token unchanged
deployment.apps/lighthouse-tekton-controller configured
rolebinding.rbac.authorization.k8s.io/lighthouse-tekton-controller unchanged
role.rbac.authorization.k8s.io/lighthouse-tekton-controller unchanged
serviceaccount/lighthouse-tekton-controller unchanged
service/lighthouse-tekton-controller unchanged
deployment.apps/lighthouse-webhooks configured
rolebinding.rbac.authorization.k8s.io/lighthouse-webhooks unchanged
role.rbac.authorization.k8s.io/lighthouse-webhooks unchanged
serviceaccount/lighthouse-webhooks unchanged
configmap/config configured
configmap/jx-install-config unchanged
configmap/plugins configured
configmap/nexus unchanged
deployment.apps/nexus-nexus configured
persistentvolumeclaim/nexus-nexus unchanged
externalsecret.kubernetes-client.io/nexus unchanged
service/nexus unchanged
externalsecret.kubernetes-client.io/tekton-container-registry-auth unchanged
externalsecret.kubernetes-client.io/tekton-container-registry-auth unchanged
service/ingress-nginx-controller-admission unchanged
configmap/ingress-nginx-controller unchanged
deployment.apps/ingress-nginx-controller configured
service/ingress-nginx-controller-metrics unchanged
poddisruptionbudget.policy/ingress-nginx-controller configured
service/ingress-nginx-controller unchanged
rolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged
role.rbac.authorization.k8s.io/ingress-nginx unchanged
serviceaccount/ingress-nginx unchanged
ingressclass.networking.k8s.io/nginx unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission configured
job.batch/ingress-nginx-admission-create unchanged
job.batch/ingress-nginx-admission-patch unchanged
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
role.rbac.authorization.k8s.io/ingress-nginx-admission unchanged
serviceaccount/ingress-nginx-admission unchanged
deployment.apps/kubernetes-external-secrets unchanged
serviceaccount/kubernetes-external-secrets unchanged
service/kubernetes-external-secrets unchanged
deployment.apps/pusher-wave-pusher-wave configured
serviceaccount/pusher-wave-pusher-wave unchanged
configmap/config-artifact-bucket unchanged
configmap/config-artifact-pvc unchanged
configmap/config-defaults unchanged
configmap/config-leader-election unchanged
configmap/config-logging unchanged
configmap/config-observability unchanged
configmap/config-registry-cert unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/config.webhook.pipeline.tekton.dev unchanged
configmap/feature-flags unchanged
configmap/pipelines-info unchanged
serviceaccount/tekton-bot configured
deployment.apps/tekton-pipelines-controller unchanged
rolebinding.rbac.authorization.k8s.io/tekton-pipelines-controller-leaderelection unchanged
rolebinding.rbac.authorization.k8s.io/tekton-pipelines-controller unchanged
role.rbac.authorization.k8s.io/tekton-pipelines-controller unchanged
serviceaccount/tekton-pipelines-controller unchanged
service/tekton-pipelines-controller unchanged
rolebinding.rbac.authorization.k8s.io/tekton-pipelines-info unchanged
role.rbac.authorization.k8s.io/tekton-pipelines-info unchanged
role.rbac.authorization.k8s.io/tekton-pipelines-leader-election unchanged
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/tekton-pipelines configured
deployment.apps/tekton-pipelines-webhook unchanged
Warning: autoscaling/v2beta1 HorizontalPodAutoscaler is deprecated in v1.22+, unavailable in v1.25+; use autoscaling/v2beta2 HorizontalPodAutoscaler
horizontalpodautoscaler.autoscaling/tekton-pipelines-webhook unchanged
rolebinding.rbac.authorization.k8s.io/tekton-pipelines-webhook-leaderelection unchanged
rolebinding.rbac.authorization.k8s.io/tekton-pipelines-webhook unchanged
role.rbac.authorization.k8s.io/tekton-pipelines-webhook unchanged
serviceaccount/tekton-pipelines-webhook unchanged
service/tekton-pipelines-webhook unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/validation.webhook.pipeline.tekton.dev unchanged
secret/webhook-certs unchanged
unable to recognize "config-root/namespaces/jx/jx-kh-check-health-checks-jx/jx-bot-token-kuberhealthycheck.yaml": no matches for kind "KuberhealthyCheck" in version "comcast.github.io/v1"
unable to recognize "config-root/namespaces/jx/jx-kh-check-health-checks-jx/jx-webhook-events-kuberhealthycheck.yaml": no matches for kind "KuberhealthyCheck" in version "comcast.github.io/v1"
unable to recognize "config-root/namespaces/jx/jx-kh-check-health-checks-jx/jx-webhook-kuberhealthycheck.yaml": no matches for kind "KuberhealthyCheck" in version "comcast.github.io/v1"
mutatingwebhookconfiguration.admissionregistration.k8s.io/webhook.pipeline.tekton.dev unchanged
make[1]: *** [versionStream/src/Makefile.mk:291: kubectl-apply] Error 1
make[1]: Leaving directory '/workspace/source'
error: failed to regenerate: failed to regenerate phase 1: failed to run 'make regen-phase-1' command in directory '.', output: ''
make: *** [versionStream/src/Makefile.mk:243: regen-check] Error 1

[ankitm123]

Disable kuberhealthy, and it will work. https://github.com/jenkins-x/terraform-aws-eks-jx/blob/4240b95efe7558de35eb195d4b3ad76fbe9d0f20/variables.tf#L475 (in tf file, set this to false). it's an upstream issue with kuberhealthy, and not jx.

[nishantn3]

@ankitm123

I have not installed kuberhealthy anyway. Since it required a remote install from terraform I disabled it.

Note: I have customized the EKS terraform code to use my custom EKS module to deploy private EKS(control plane private)

[nishantn3]

@ankitm123

Hi,

I was able to make some progress. Now stuck at this

error: failed to find hmac token from secret: could not find lighthouse hmac token lighthouse-hmac-token in namespace jx: secrets "lighthouse-hmac-token" not found
make: *** [versionStream/src/Makefile.mk:216: gitops-webhook-update] Error 1

Can you help?

[nishantn3]

Closing, I could fix and deploy Jenkins-X. As I was not using the terraform repo for eks by jenkins-x I was facing some issues that I have figured out