search

argoproj / argo-rollouts

Progressive Delivery for Kubernetes
https://argo-rollouts.readthedocs.io/
Apache License 2.0
2.75k stars 864 forks source link

Argo rollout not synchronizing new git image change after RolloutAborted state #3578

Open Cstack-Krish opened 5 months ago

Cstack-Krish commented 5 months ago

Describe the bug

We are using argo rollouts for canary deployment. In between a step in the canary deployment, we abort the stage and Rollout object goes into the Degraded state and the canary deployment gets aborted as expected. But after this when we update the image of the deployment, argocd is having issues synchronizing the new changes. It fails to synchronize the change and after 5 unsuccessful attempts the application sync status goes to Sync failed with the 1 Error status showing Failed sync attempt one or more synchronization tasks completed unsuccessfully (retried 5 times).

To Reproduce

Define a rollout object having a workloadRef to the deployment object. Set the strategy as Canary and configure the required canary deployment strategy parameters. Then after an image update to the deployment object from A to B, the rollout object creates a canary service which would have the pod with image B and the stable service would be with pod A. For example 20% of traffic would be in canary service and rest in stable service. We then Abort the rollout, which will lead to the termination of canary service with image B. At this time the rollout object is in RolloutAborted status. Then when we change the image of deployment object to image C, argo fails to sync this change to the deployment. that is, it fails to synchronize the change and after 5 unsuccessful attempts the application sync status goes to Sync failed.

Expected behavior

We expect argo to sync the deployment with the latest changes in git, that is, to Image C.

Screenshots

image image

Version

Argo-Rollouts Version: v1.6.6 AgoCD Version: v2.11.0

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

kostis-codefresh commented 5 months ago

Argo Rollouts has no knowledge of Git. And it never syncs anything. If this is a bug, it should be opened in the Argo CD Github.

Cstack-Krish commented 5 months ago

The mentioned issue was observed when we used the WorkloadRef to the deployment rather than using the Rollout template property in the Rollout manifest. There wasn't any issue observed in the synchronization of git changes when we changed the template to use the Rollout template property and not the WorkloadRef. So we believe this has something to do with argo-rollouts rather than argo-cd. image