bookingcom / shipper

Kubernetes native multi-cluster canary or blue-green rollouts using Helm
Apache License 2.0
733 stars 39 forks source link

Historical releases preserve a once-obtained ClustersNotReady condition #299

Closed osdrv closed 4 years ago

osdrv commented 4 years ago

It seems historical releases never get rid of a once-obtained ClustersNotReady strategy condition even if the condition is gone. Based on the user feedback, this is primarily a problem on historical releases, not on contender/incumbent pair.

Example of a condition:

status:
  strategy:
    conditions:
    - lastTransitionTime: "2020-03-11T16:33:00Z"
      message: 'release "app-7acaa8df-0" hasn''t achieved capacity in clusters: foo:
        ; bar: . for more details try `kubectl describe ct app-7acaa8df-0`'
      reason: ClustersNotReady
      status: "False"
      step: 2
      type: ContenderAchievedCapacity

From shipper logs:

I0313 14:04:28.877040       1 scheduler.go:91] Processing release "foobar/app-7acaa8df-0"
I0313 14:04:28.899010       1 scheduler.go:585] Extracted 1 replicas from release "foobar/app-7acaa8df-0"
I0313 14:04:28.899074       1 scheduler.go:136] Finished processing "foobar/app-7acaa8df-0"
I0313 14:04:28.899107       1 release_controller.go:349] Release "foobar/app-7acaa8df-0" has been successfully scheduled
I0313 14:04:28.899203       1 strategy_executor.go:242] Release "foobar/app-7acaa8df-0" has achieved capacity
I0313 14:04:28.899222       1 strategy_executor.go:313] Release "foobar/app-7acaa8df-0" has achieved traffic
I0313 14:04:28.899230       1 release_controller.go:468] Strategy verified for release "foobar/app-7acaa8df-0", nothing to patch
I0313 14:04:28.899454       1 release_controller.go:386] Done processing Release "foobar/app-7acaa8df-0"
I0313 14:04:28.899470       1 release_controller.go:245] Successfully synced Release "foobar/app-7acaa8df-0"
osdrv commented 4 years ago

Fixed in 0.8.2