jenkinsci / google-kubernetes-engine-plugin

The Google Kubernetes Engine (GKE) Plugin allows you to deploy build artifacts to Kubernetes clusters running in GKE with Jenkins.
https://plugins.jenkins.io/google-kubernetes-engine/
Apache License 2.0
36 stars 66 forks source link

"Verify Deployments" does not work for non-default namespace #297

Open wadexu007 opened 1 year ago

wadexu007 commented 1 year ago

Jenkins and plugins versions report

Environment ```text Google Kubernetes Engine Plugin 0.8.7 Jenkins 2.361.3 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

GKE 1.22 Google Kubernetes Engine Plugin 0.8.7 Jenkins 2.361.3

Reproduction steps

This is work when my deployment.yaml deploy to a default namespace.

                step([
                  $class: 'KubernetesEngineBuilder',
                  projectId: env.PROJECT_ID,
                  clusterName: env.CLUSTER_NAME,
                  location: env.REGION,
                  manifestPattern: 'deployment.yaml',
                  credentialsId: env.CREDENTIALS_ID,
                  verifyDeployments: true])

But once my deployment in a non-default namespace, Verifying: apps/v1/Deployment: demo-app timeout till throws errors:

Failed to verify apps/v1/Deployment: demo-app
java.io.IOException: Failed to launch command args: [kubectl, --kubeconfig, /home/jenkins/agent/workspace/multibranch-pipeline-demo_main/Jenkins/k8s_pod_as_build_agent/demo-app-go@tmp/.kube13763572759526201163config, get, deployment, demo-app, -o, json], status: 1. Logs: Error from server (NotFound): deployments.apps "demo-app" not found

    at com.google.jenkins.plugins.k8sengine.KubectlWrapper.launchAndJoinCommand(KubectlWrapper.java:174)
    at com.google.jenkins.plugins.k8sengine.KubectlWrapper.runKubectlCommand(KubectlWrapper.java:138)
    at com.google.jenkins.plugins.k8sengine.KubectlWrapper.getObject(KubectlWrapper.java:192)
    at com.google.jenkins.plugins.k8sengine.KubernetesVerifiers$DeploymentVerifier.verify(KubernetesVerifiers.java:162)
    at com.google.jenkins.plugins.k8sengine.KubernetesVerifiers.verify(KubernetesVerifiers.java:225)
    at com.google.jenkins.plugins.k8sengine.VerificationTask.verify(VerificationTask.java:80)
    at com.google.jenkins.plugins.k8sengine.VerificationTask.lambda$verifyObjects$5(VerificationTask.java:119)
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)
    at reactor.core.publisher.FluxFilter$FilterSubscriber.onNext(FluxFilter.java:113)
    at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99)
    at reactor.core.publisher.FluxRepeatWhen$RepeatWhenMainSubscriber.onNext(FluxRepeatWhen.java:142)
    at reactor.core.publisher.FluxIterable$IterableSubscription.fastPath(FluxIterable.java:338)
    at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:225)
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2193)
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2067)
    at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:164)
    at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86)
    at reactor.core.publisher.FluxRepeatWhen$RepeatWhenMainSubscriber.resubscribe(FluxRepeatWhen.java:184)
    at reactor.core.publisher.FluxRepeatWhen$RepeatWhenOtherSubscriber.onNext(FluxRepeatWhen.java:239)
    at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.innerNext(FluxConcatMap.java:281)
    at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:860)
    at reactor.core.publisher.MonoDelay$MonoDelayRunnable.run(MonoDelay.java:119)
    at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
    at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

Expected Results

Verified successfully same as I use default namespace.

Actual Results

Failed to verify apps/v1/Deployment: demo-app

Anything else?

my deployment.yaml is easy.

apiVersion: v1
kind: Namespace
metadata:
  labels:
    app: demo-app
  name: demo
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: demo-app
    managed-by: Kustomize
  name: demo-app
  namespace: demo
---
apiVersion: v1
data:
  config.json: |-
    {
        "SOME_CONFIG": "/demo/path"
    }
kind: ConfigMap
metadata:
  labels:
    app: demo-app
  name: demo-app-config-t7c64mbtt2
  namespace: demo
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: demo-app
    managed-by: Kustomize
  name: demo-app
  namespace: demo
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: http
  selector:
    app: demo-app
    managed-by: Kustomize
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: demo-app
    managed-by: Kustomize
  name: demo-app
  namespace: demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-app
      managed-by: Kustomize
  template:
    metadata:
      labels:
        app: demo-app
        managed-by: Kustomize
    spec:
      containers:
      - image: wadexu007/demo-app:1.0.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /pizzas
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 40
          timeoutSeconds: 1
        name: app
        ports:
        - containerPort: 8080
          name: http
        readinessProbe:
          failureThreshold: 5
          httpGet:
            path: /pizzas
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 20
          timeoutSeconds: 1
        resources:
          limits:
            cpu: 1
            memory: 1Gi
          requests:
            cpu: 200m
            memory: 256Mi
        securityContext:
          allowPrivilegeEscalation: false
        volumeMounts:
        - mountPath: /app/conf/config.json
          name: config-volume
          subPath: config.json
      serviceAccountName: demo-app
      volumes:
      - configMap:
          name: demo-app-config-t7c64mbtt2
        name: config-volume
bangupoc commented 1 year ago

Facing a similar error:

Failed to verify apps/v1/Deployment: app-test java.io.IOException: Failed to launch command args: [kubectl, --kubeconfig, /var/lib/jenkins/workspace/sample_dev_gke_deploy@tmp/.kube14897892684774139622config, get, deployment, catalog-management, -o, json], status: 1. Logs: Error from server (NotFound): deployments.apps "app-test" not found

Is this issue resolved still existing ? As per our understanding deployment happens but jenkins plugin is not able to verify?

So it is safe to by pass this error till the issue gets fixed ?

bangupoc commented 1 year ago

Hi @wadexu007 were you able to fix this error we are also facing a similar issue

wadexu007 commented 1 year ago

@bangupoc Not fix. disable it currently. verifyDeployments: false

ViliusS commented 1 year ago

@wadexu007 you are not setting the namespace in your step arguments. Works for me with:

     step([
                $class: 'KubernetesEngineBuilder',
                projectId: params.project,
                clusterName: env.CLUSTER_NAME,
                location: env.LOCATION,
                namespace: env.NAMESPACE,
                manifestPattern: 'newDeploy.yaml',
                credentialsId: env.GCP_CREDENCIALS_ID,
                verifyDeployments: true,
                verifyTimeoutInMinutes: 10
     ])