eclipse-symphony / symphony

Symphony project
MIT License
37 stars 29 forks source link

K8S target provider cannot report the deployment status correctly #344

Open msftcoderdjw opened 3 months ago

msftcoderdjw commented 3 months ago

Current k8s target provider only check the error code of creating deployment while deployment creation is an async operation. If users' template has wrong invalid image name, the deployment creation would be successful but the deployment will fail at last.

We need to monitor the deployment status then mark the instance/target state.

Like:

        if deployment.Status.UpdatedReplicas == *deployment.Spec.Replicas &&
            deployment.Status.Replicas == *deployment.Spec.Replicas &&
            deployment.Status.AvailableReplicas == *deployment.Spec.Replicas &&
            deployment.Status.ObservedGeneration >= deployment.Generation {
            // success
        }
kubectl get deployment -n line-1 -o yaml
apiVersion: v1
items:
- apiVersion: apps/v1
  kind: Deployment
  metadata:
    annotations:
      deployment.kubernetes.io/revision: "2"
    creationTimestamp: "2024-07-11T13:54:28Z"
    generation: 2
    name: sampleinstance-v1
    namespace: line-1
    resourceVersion: "30250"
    uid: e7b10e8f-8323-4087-8211-575dfcb1d48a
  spec:
    progressDeadlineSeconds: 600
    replicas: 1
    revisionHistoryLimit: 10
    selector:
      matchLabels:
        app: sampleinstance-v1
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        creationTimestamp: null
        labels:
          app: sampleinstance-v1
      spec:
        containers:
        - image: ${$config('samplecatalog:v1','image')}
          imagePullPolicy: Always
          name: sample-prometheus-server
          ports:
          - containerPort: 9090
            protocol: TCP
          resources:
            requests:
              cpu: 100m
              memory: 100Mi
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 30
  status:
    availableReplicas: 1
    conditions:
    - lastTransitionTime: "2024-07-11T13:54:35Z"
      lastUpdateTime: "2024-07-11T13:54:35Z"
      message: Deployment has minimum availability.
      reason: MinimumReplicasAvailable
      status: "True"
      type: Available
    - lastTransitionTime: "2024-07-11T13:54:28Z"
      lastUpdateTime: "2024-07-11T14:06:29Z"
      message: ReplicaSet "sampleinstance-v1-b4b67ddc7" is progressing.
      reason: ReplicaSetUpdated
      status: "True"
      type: Progressing
    observedGeneration: 2
    readyReplicas: 1
    replicas: 2
    unavailableReplicas: 1
    updatedReplicas: 1
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""
msftcoderdjw commented 3 months ago

@LeiQL to improve this. We can consider add wait option to K8S target provider.

msftcoderdjw commented 1 month ago

Reassign this to @FireDefend

FireDefend commented 2 weeks ago

fixed by add wait operation