knative-extensions / serving-progressive-rollout

Knative Serving extension to roll out the revision progressively
Apache License 2.0
3 stars 6 forks source link

NewObservedGenFailure will pop up once upgrading the images of serving progressive rollout #193

Closed houshengbo closed 1 month ago

houshengbo commented 2 months ago

Function-wise, it won't bring any blocker, but this status can show up in the CR status of RO.

 - lastTransitionTime: "2024-05-23T02:05:46Z"
    message: unsuccessfully observed a new generation
    reason: NewObservedGenFailure
    status: Unknown
    type: Ready
houshengbo commented 2 months ago
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: autoscale-go
  namespace: default
spec:
  template:
    metadata:
      annotations:
        # Knative concurrency-based autoscaling (default).
        autoscaling.knative.dev/class: kpa.autoscaling.knative.dev
        autoscaling.knative.dev/metric: concurrency
        # Target 10 requests in-flight per pod.
        autoscaling.knative.dev/target: "10"
        # Disable scale to zero with a min scale of 1.
        autoscaling.knative.dev/min-scale: "1"
        #autoscaling.knative.dev/max-scale: "5"
        rollout.knative.dev/progressive-rollout-enabled: "true"
        #rollout.knative.dev/progressive-rollout-strategy: "availability"
        rollout.knative.dev/progressive-rollout-strategy: "resourceUtil"
    spec:
      containers:
        - image: ghcr.io/knative/autoscale-go:latest

then

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: autoscale-go
  namespace: default
spec:
  template:
    metadata:
      annotations:
        # Knative concurrency-based autoscaling (default).
        autoscaling.knative.dev/class: kpa.autoscaling.knative.dev
        autoscaling.knative.dev/metric: concurrency
        # Target 10 requests in-flight per pod.
        autoscaling.knative.dev/target: "1"
        # Disable scale to zero with a min scale of 1.
        autoscaling.knative.dev/min-scale: "1"
        # Limit scaling to 100 pods.
        autoscaling.knative.dev/max-scale: "6"
        rollout.knative.dev/progressive-rollout-enabled: "true"
        #rollout.knative.dev/progressive-rollout-strategy: "availability"
        rollout.knative.dev/progressive-rollout-strategy: "resourceUtil"
    spec:
      containers:
        #- image: artprod.dev.bloomberg.com/ds/shou73/autoscale-go:latest
        - image: ghcr.io/knative/autoscale-go:latest1

then remove the ro and let it recreate the ro. The status will be marked NewObservedGenFailure.

This is one way to recreate. There are other ways, but not convenient at all.

houshengbo commented 2 months ago
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: autoscale-go
  namespace: default
spec:
  template:
    metadata:
      annotations:
        # Knative concurrency-based autoscaling (default).
        autoscaling.knative.dev/class: kpa.autoscaling.knative.dev
        autoscaling.knative.dev/metric: concurrency
        # Target 10 requests in-flight per pod.
        autoscaling.knative.dev/target: "10"
        # Disable scale to zero with a min scale of 1.
        autoscaling.knative.dev/min-scale: "1"
        #autoscaling.knative.dev/max-scale: "5"
        rollout.knative.dev/progressive-rollout-enabled: "true"
        #rollout.knative.dev/progressive-rollout-strategy: "availability"
        rollout.knative.dev/progressive-rollout-strategy: "resourceUtil"
    spec:
      containers:
        - image: ghcr.io/knative/autoscale-go:latest

then

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: autoscale-go
  namespace: default
spec:
  template:
    metadata:
      annotations:
        # Knative concurrency-based autoscaling (default).
        autoscaling.knative.dev/class: kpa.autoscaling.knative.dev
        autoscaling.knative.dev/metric: concurrency
        # Target 10 requests in-flight per pod.
        autoscaling.knative.dev/target: "1"
        # Disable scale to zero with a min scale of 1.
        autoscaling.knative.dev/min-scale: "1"
        # Limit scaling to 100 pods.
        autoscaling.knative.dev/max-scale: "6"
        rollout.knative.dev/progressive-rollout-enabled: "true"
        rollout.knative.dev/progressive-rollout-strategy: "resourceUtil"
    spec:
      containers:
        - image: ghcr.io/knative/autoscale-go:latest1

then remove the ro and let it recreate the ro. The status will be marked NewObservedGenFailure.

This is one way to recreate. There are other ways, but not convenient at all.

houshengbo commented 2 months ago

The reconcile loop should update the status of the CR, no matter it really has any changes on the status.