Argocd ignores retry options: disabled

[fabry00]

Describe the bug

I have an ArgoCD Application and from ArgoCD UI I see that the RETRY OPTIONS is "Retry disabled"

I have a Job with a pre-sync hook and the backoff limit of the Job is 2

If the Job fails I see that ArgoCd is ignoring the retry options and it continues to sync (deletes the job and redeploy it) for 10 minutes

I need ArgoCD to mark the Sync as failed as soon as the job fails, but it took more than 10 minutes to ArgoCD to stop synchronizing the current commit which was making the Job failing and start synchronizing the last commit with the fix.

In the image below, you can see that ArgoCD has already tried to Sync 5 times the application even if the Retry is disabled and there is new commit waiting to be sync (this second commit should fix the job issue)

Application:

To Reproduce

Create an application.yaml CR

    project: xxxxx
    source:
      repoURL: >-
       xxxx
      path: helm
      targetRevision: xxxx
      helm:
        valueFiles:
          - xxx
          - xxxx
        parameters:
          - name: argo.build_env.ARGOCD_APP_REVISION
            value: $ARGOCD_APP_REVISION
          - name: argo.build_env.ARGOCD_APP_SOURCE_REPO_URL
            value: $ARGOCD_APP_SOURCE_REPO_URL
          - name: argo.build_env.ARGOCD_APP_SOURCE_TARGET_REVISION
            value: $ARGOCD_APP_SOURCE_TARGET_REVISION
          - name: argo.build_env.ARGOCD_APP_SOURCE_PATH
            value: $ARGOCD_APP_SOURCE_PATH
        skipCrds: true
    destination:
      server: 'https://kubernetes.default.svc'
      namespace: xxxxx
    syncPolicy:
      automated:
        prune: true
        selfHeal: true
      syncOptions:
        - FailOnSharedResource=true
        - RespectIgnoreDifferences=true
    ignoreDifferences:
      - kind: ConfigMap
        name: xxxx
        jqPathExpressions:
          - .data

Create a Job in your helm char with

annotations:
   argocd.argoproj.io/hook: Sync
   argocd.argoproj.io/hook-delete-policy: BeforeHookCreation

Commit something that let the job fails, as soon as argoCD start the sync, revert the previous commit in order to fix the issue. ArgoCD will perform multiple retries even if the retry is disabled

Expected behavior

Fail the sync as soon as the job fail and pick up the next commit in the queue

Version

argocd: v2.8.4+c279299
  BuildDate: 2023-09-13T19:12:09Z
  GitCommit: c27929928104dc37b937764baf65f38b78930e59
  GitTreeState: clean
  GoVersion: go1.20.6
  Compiler: gc
  Platform: linux/amd64

Logs

time="2023-09-21T19:09:39Z" level=info msg="Applying resource Job/myjob in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:39Z" level=info msg="Applying resource CronJob/certificate-exporter in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:39Z" level=info msg="Updating operation state. phase: Running -> Running, message: 'one or more synchronization tasks completed unsuccessfully. Retrying attempt #5 at 7:08PM.' -> 'one or more tasks are running'" application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00166-PNpGr
time="2023-09-21T19:09:39Z" level=info msg="Applying resource ExternalSecret/secret in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:39Z" level=info msg="Applying resource ExternalSecret/git-access-token in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:39Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'externalsecret.external-secrets.io/secret configured'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ExternalSecret name=secret namespace=myapp-controlplane-dev phase=Sync syncId=00166-PNpGr
time="2023-09-21T19:09:39Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'externalsecret.external-secrets.io/git-access-token configured'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ExternalSecret name=git-access-token namespace=myapp-controlplane-dev phase=Sync syncId=00166-PNpGr
time="2023-09-21T19:09:41Z" level=info msg="Updating operation state. phase: Running -> Running, message: 'one or more tasks are running' -> 'waiting for healthy state of /ServiceAccount/certificate-exporter and 4 more resources'" application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00166-PNpGr
time="2023-09-21T19:09:41Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=3.924470729s syncId=00166-PNpGr
time="2023-09-21T19:09:42Z" level=info msg="updated 'api-argocd-dev/myapp-controlplane-configs-dev' operation (phase: Running)" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Resuming in-progress operation. phase: Running, message: waiting for healthy state of /ServiceAccount/certificate-exporter and 4 more resources" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/bootstrap-cluster-api-platform-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=9 unmarshal_ms=9 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=7 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00167-FvnHc
time="2023-09-21T19:09:42Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00167-FvnHc tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Running,externalsecret.external-secrets.io/secret configured), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Running,externalsecret.external-secrets.io/git-access-token configured), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (,,), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (,,), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (,,), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,,), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (,,)]"
time="2023-09-21T19:09:42Z" level=info msg="Updating resource result, status: 'Synced' -> 'Synced', phase 'Running' -> 'Succeeded', message 'externalsecret.external-secrets.io/secret configured' -> 'Secret was synced'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ExternalSecret name=secret namespace=myapp-controlplane-dev phase=Sync syncId=00167-FvnHc
time="2023-09-21T19:09:42Z" level=info msg="Updating resource result, status: 'Synced' -> 'Synced', phase 'Running' -> 'Succeeded', message 'externalsecret.external-secrets.io/git-access-token configured' -> 'Secret was synced'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ExternalSecret name=git-access-token namespace=myapp-controlplane-dev phase=Sync syncId=00167-FvnHc
time="2023-09-21T19:09:42Z" level=info msg="Updating operation state. phase: Running -> Running, message: 'waiting for healthy state of /ServiceAccount/certificate-exporter and 4 more resources' -> 'one or more tasks are running'" application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00167-FvnHc
time="2023-09-21T19:09:42Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/bootstrap-cluster-api-platform-dev dedup_ms=0 dest-name= dest-namespace=api-argocd-dev dest-server="https://kubernetes.default.svc" diff_ms=20 fields.level=1 git_ms=9 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=51
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=51.41836ms syncId=00167-FvnHc
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=7 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg="Skipping auto-sync: another operation is in progress" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dedup_ms=0 dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" diff_ms=13 fields.level=1 git_ms=7 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=32
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (0)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" fields.level=0 time_ms=6
time="2023-09-21T19:09:42Z" level=info msg="updated 'api-argocd-dev/myapp-controlplane-configs-dev' operation (phase: Running)" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (0)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Resuming in-progress operation. phase: Running, message: one or more tasks are running" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" fields.level=0 time_ms=5
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (0)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/bootstrap-cluster-api-platform-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=9 unmarshal_ms=9 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=7 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00168-EQaPA
time="2023-09-21T19:09:42Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00168-EQaPA tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (,,), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (,,), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (,,), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,,), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (,,)]"
time="2023-09-21T19:09:42Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/bootstrap-cluster-api-platform-dev dedup_ms=0 dest-name= dest-namespace=api-argocd-dev dest-server="https://kubernetes.default.svc" diff_ms=20 fields.level=1 git_ms=9 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=50
time="2023-09-21T19:09:42Z" level=info msg="Update successful" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" fields.level=0 time_ms=49
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=27.827597ms syncId=00168-EQaPA
time="2023-09-21T19:09:42Z" level=info msg="No operation updates necessary to 'api-argocd-dev/myapp-controlplane-configs-dev'. Skipping patch" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/bootstrap-cluster-api-platform-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=8 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg="Resuming in-progress operation. phase: Running, message: one or more tasks are running" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=7 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/bootstrap-cluster-api-platform-dev dedup_ms=0 dest-name= dest-namespace=api-argocd-dev dest-server="https://kubernetes.default.svc" diff_ms=19 fields.level=1 git_ms=8 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=46
time="2023-09-21T19:09:42Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00169-sxnDV
time="2023-09-21T19:09:42Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00169-sxnDV tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (,,), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (,,), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (,,), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,,), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (,,)]"
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=7 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:09:42Z" level=info msg="Applying resource ServiceAccount/certificate-exporter in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:42Z" level=info msg="Skipping auto-sync: another operation is in progress" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Update successful" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:42Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dedup_ms=0 dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" diff_ms=13 fields.level=1 git_ms=7 health_ms=1 live_ms=0 settings_ms=0 sync_ms=0 time_ms=479
time="2023-09-21T19:09:42Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:42Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/bootstrap-cluster-api-platform-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=100 unmarshal_ms=99 version_ms=0
time="2023-09-21T19:09:43Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:43Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:43Z" level=info msg="Reconciliation completed" application=api-argocd-dev/bootstrap-cluster-api-platform-dev dedup_ms=0 dest-name= dest-namespace=api-argocd-dev dest-server="https://kubernetes.default.svc" diff_ms=199 fields.level=1 git_ms=100 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=618
time="2023-09-21T19:09:43Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/myapp-controlplane-dev
time="2023-09-21T19:09:43Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-dev
time="2023-09-21T19:09:43Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=13 unmarshal_ms=12 version_ms=0
time="2023-09-21T19:09:44Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'serviceaccount/certificate-exporter unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ServiceAccount name=certificate-exporter namespace=myapp-controlplane-dev phase=Sync syncId=00169-sxnDV
time="2023-09-21T19:09:44Z" level=info msg="Applying resource ConfigMap/myapp-controlplane-configs-checksums in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:44Z" level=info msg="Applying resource ConfigMap/certificate-exporter in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:44Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'configmap/certificate-exporter unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ConfigMap name=certificate-exporter namespace=myapp-controlplane-dev phase=Sync syncId=00169-sxnDV
time="2023-09-21T19:09:44Z" level=info msg="Refreshing app status (comparison expired, requesting refresh. reconciledAt: 2023-09-21 19:08:42 +0000 UTC, expiry: 1m0s), level (2)" application=api-argocd-dev/myapp-dataplane-common-dev
time="2023-09-21T19:09:44Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-dataplane-dev)" application=api-argocd-dev/myapp-dataplane-common-dev
time="2023-09-21T19:09:44Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'configmap/myapp-controlplane-configs-checksums unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ConfigMap name=myapp-controlplane-configs-checksums namespace=myapp-controlplane-dev phase=Sync syncId=00169-sxnDV
time="2023-09-21T19:09:44Z" level=info msg="Applying resource Job/myjob in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:44Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-dataplane-common-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=65 unmarshal_ms=65 version_ms=0
time="2023-09-21T19:09:44Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:44Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:44Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'job.batch/myjob created'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=Job name=myjob namespace=myapp-controlplane-dev phase=Sync syncId=00169-sxnDV
time="2023-09-21T19:09:44Z" level=info msg="Applying resource CronJob/certificate-exporter in cluster: https://172.21.0.1:443, namespace: myapp-controlplane-dev"
time="2023-09-21T19:09:44Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/myapp-controlplane-dev
time="2023-09-21T19:09:44Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/myapp-controlplane-dev
time="2023-09-21T19:09:44Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-dev dedup_ms=0 dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" diff_ms=792 fields.level=1 git_ms=13 health_ms=6 live_ms=1 settings_ms=0 sync_ms=0 time_ms=896
time="2023-09-21T19:09:44Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'cronjob.batch/certificate-exporter unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=CronJob name=certificate-exporter namespace=myapp-controlplane-dev phase=Sync syncId=00169-sxnDV
time="2023-09-21T19:09:44Z" level=info msg="Updating operation state. phase: Running -> Running, message: 'one or more tasks are running' -> 'waiting for completion of hook batch/Job/myjob'" application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00169-sxnDV
time="2023-09-21T19:09:44Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=2.589319199s syncId=00169-sxnDV
time="2023-09-21T19:09:44Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=8 unmarshal_ms=8 version_ms=0
time="2023-09-21T19:09:44Z" level=info msg="updated 'api-argocd-dev/myapp-controlplane-configs-dev' operation (phase: Running)" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:09:44Z" level=info msg="Skipping auto-sync: another operation is in progress" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:44Z" level=info msg="Resuming in-progress operation. phase: Running, message: waiting for completion of hook batch/Job/myjob" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:44Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:44Z" level=info msg="Refreshing app status (comparison expired, requesting refresh. reconciledAt: 2023-09-21 19:08:43 +0000 UTC, expiry: 1m0s), level (2)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:44Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:44Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=10 unmarshal_ms=9 version_ms=0
time="2023-09-21T19:09:44Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00170-BicWH
time="2023-09-21T19:09:44Z" level=info msg="Update successful" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:44Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dedup_ms=0 dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" diff_ms=13 fields.level=1 git_ms=8 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=64
time="2023-09-21T19:09:44Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00170-BicWH tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Running,serviceaccount/certificate-exporter unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (Synced,Running,configmap/myapp-controlplane-configs-checksums unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Running,configmap/certificate-exporter unchanged), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (Synced,Running,job.batch/myjob created), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Running,cronjob.batch/certificate-exporter unchanged)]"
time="2023-09-21T19:09:44Z" level=info msg="Updating resource result, status: 'Synced' -> 'Synced', phase 'Running' -> 'Succeeded', message 'serviceaccount/certificate-exporter unchanged' -> 'serviceaccount/certificate-exporter unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ServiceAccount name=certificate-exporter namespace=myapp-controlplane-dev phase=Sync syncId=00170-BicWH
time="2023-09-21T19:09:44Z" level=info msg="Updating resource result, status: 'Synced' -> 'Synced', phase 'Running' -> 'Succeeded', message 'configmap/myapp-controlplane-configs-checksums unchanged' -> 'configmap/myapp-controlplane-configs-checksums unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ConfigMap name=myapp-controlplane-configs-checksums namespace=myapp-controlplane-dev phase=Sync syncId=00170-BicWH
time="2023-09-21T19:09:44Z" level=info msg="Updating resource result, status: 'Synced' -> 'Synced', phase 'Running' -> 'Succeeded', message 'configmap/certificate-exporter unchanged' -> 'configmap/certificate-exporter unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=ConfigMap name=certificate-exporter namespace=myapp-controlplane-dev phase=Sync syncId=00170-BicWH
time="2023-09-21T19:09:44Z" level=info msg="Updating resource result, status: 'Synced' -> '', phase 'Running' -> 'Running', message 'job.batch/myjob created' -> 'job.batch/myjob created'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=Job name=myjob namespace=myapp-controlplane-dev phase=Sync syncId=00170-BicWH
time="2023-09-21T19:09:44Z" level=info msg="Updating resource result, status: 'Synced' -> 'Synced', phase 'Running' -> 'Succeeded', message 'cronjob.batch/certificate-exporter unchanged' -> 'cronjob.batch/certificate-exporter unchanged'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=CronJob name=certificate-exporter namespace=myapp-controlplane-dev phase=Sync syncId=00170-BicWH
time="2023-09-21T19:09:44Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=8.562503ms syncId=00170-BicWH
time="2023-09-21T19:09:45Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/bootstrap-cluster-api-platform-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=169 unmarshal_ms=169 version_ms=0
time="2023-09-21T19:09:45Z" level=info msg="updated 'api-argocd-dev/myapp-controlplane-configs-dev' operation (phase: Running)" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Resuming in-progress operation. phase: Running, message: waiting for completion of hook batch/Job/myjob" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=99 plugins_ms=0 repo_ms=0 time_ms=202 unmarshal_ms=101 version_ms=0
time="2023-09-21T19:09:45Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=100 plugins_ms=0 repo_ms=0 time_ms=304 unmarshal_ms=203 version_ms=0
time="2023-09-21T19:09:45Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00171-ARPgM
time="2023-09-21T19:09:45Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00171-ARPgM tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,serviceaccount/certificate-exporter unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (Synced,Succeeded,configmap/myapp-controlplane-configs-checksums unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,configmap/certificate-exporter unchanged), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,Running,job.batch/myjob created), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,cronjob.batch/certificate-exporter unchanged)]"
time="2023-09-21T19:09:45Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=9.505314ms syncId=00171-ARPgM
time="2023-09-21T19:09:45Z" level=info msg="Skipping auto-sync: another operation is in progress" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg="No operation updates necessary to 'api-argocd-dev/myapp-controlplane-configs-dev'. Skipping patch" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Resuming in-progress operation. phase: Running, message: waiting for completion of hook batch/Job/myjob" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Update successful" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg="Reconciliation completed" application=api-argocd-dev/bootstrap-cluster-api-platform-dev dedup_ms=0 dest-name= dest-namespace=api-argocd-dev dest-server="https://kubernetes.default.svc" diff_ms=414 fields.level=2 git_ms=170 health_ms=0 live_ms=2 settings_ms=0 sync_ms=0 time_ms=670
time="2023-09-21T19:09:45Z" level=info msg="Refreshing app status (comparison expired, requesting refresh. reconciledAt: 2023-09-21 19:08:43 +0000 UTC, expiry: 1m0s), level (2)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg="Update successful" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dedup_ms=0 dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" diff_ms=17 fields.level=1 git_ms=304 health_ms=0 live_ms=0 settings_ms=95 sync_ms=0 time_ms=496
time="2023-09-21T19:09:45Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg="Refreshing app status (controller refresh requested), level (0)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" fields.level=0 time_ms=6
time="2023-09-21T19:09:45Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=8 unmarshal_ms=8 version_ms=0
time="2023-09-21T19:09:45Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/bootstrap-cluster-api-platform-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=9 unmarshal_ms=9 version_ms=0
time="2023-09-21T19:09:45Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/myapp-dataplane-common-dev
time="2023-09-21T19:09:45Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00172-Oucxz
time="2023-09-21T19:09:45Z" level=info msg="Update successful" application=api-argocd-dev/myapp-dataplane-common-dev
time="2023-09-21T19:09:45Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-dataplane-common-dev dedup_ms=0 dest-name= dest-namespace=myapp-dataplane-dev dest-server="https://kubernetes.default.svc" diff_ms=136 fields.level=2 git_ms=65 health_ms=3 live_ms=0 settings_ms=0 sync_ms=0 time_ms=900
time="2023-09-21T19:09:45Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00172-Oucxz tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,serviceaccount/certificate-exporter unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (Synced,Succeeded,configmap/myapp-controlplane-configs-checksums unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,configmap/certificate-exporter unchanged), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,Running,job.batch/myjob created), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,cronjob.batch/certificate-exporter unchanged)]"
time="2023-09-21T19:09:45Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=8.640569ms syncId=00172-Oucxz
time="2023-09-21T19:09:45Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/myapp-dataplane-common-dev
time="2023-09-21T19:09:45Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-dataplane-dev)" application=api-argocd-dev/myapp-dataplane-common-dev
time="2023-09-21T19:09:45Z" level=info msg="No operation updates necessary to 'api-argocd-dev/myapp-controlplane-configs-dev'. Skipping patch" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-dataplane-common-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=7 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:09:45Z" level=info msg="Resuming in-progress operation. phase: Running, message: waiting for completion of hook batch/Job/myjob" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:09:45Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=9 unmarshal_ms=9 version_ms=0
time="2023-09-21T19:09:45Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00173-bUkYc
time="2023-09-21T19:09:45Z" level=info msg="Update successful" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg="Reconciliation completed" application=api-argocd-dev/bootstrap-cluster-api-platform-dev dedup_ms=0 dest-name= dest-namespace=api-argocd-dev dest-server="https://kubernetes.default.svc" diff_ms=22 fields.level=2 git_ms=9 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=75
time="2023-09-21T19:09:45Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:09:45Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00173-bUkYc tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,serviceaccount/certificate-exporter unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (Synced,Succeeded,configmap/myapp-controlplane-configs-checksums unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,configmap/certificate-exporter unchanged), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,Running,job.batch/myjob created), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,cronjob.batch/certificate-exporter unchanged)]"
time="2023-09-21T19:09:45Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=13.731197ms syncId=00173-bUkYc
...
time="2023-09-21T19:10:28Z" level=info msg="Resuming in-progress operation. phase: Running, message: waiting for completion of hook batch/Job/myjob" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:28Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:28Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=6 unmarshal_ms=6 version_ms=0
time="2023-09-21T19:10:28Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00199-ljFUr
time="2023-09-21T19:10:28Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00199-ljFUr tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,serviceaccount/certificate-exporter unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (Synced,Succeeded,configmap/myapp-controlplane-configs-checksums unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,configmap/certificate-exporter unchanged), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,Running,job.batch/myjob created), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,cronjob.batch/certificate-exporter unchanged)]"
time="2023-09-21T19:10:28Z" level=info msg="Updating resource result, status: '' -> '', phase 'Running' -> 'Failed', message 'job.batch/myjob created' -> 'Job has reached the specified backoff limit'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=Job name=myjob namespace=myapp-controlplane-dev phase=Sync syncId=00199-ljFUr
time="2023-09-21T19:10:28Z" level=info msg="Updating operation state. phase: Running -> Failed, message: 'waiting for completion of hook batch/Job/myjob' -> 'one or more synchronization tasks completed unsuccessfully'" application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00199-ljFUr
time="2023-09-21T19:10:28Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=8.722162ms syncId=00199-ljFUr
time="2023-09-21T19:10:28Z" level=info msg="updated 'api-argocd-dev/myapp-controlplane-configs-dev' operation (phase: Failed)" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:10:28Z" level=info msg="Sync operation to bbd3b2e9b61346ca4115e26128d82febb7ec4e5a failed: one or more synchronization tasks completed unsuccessfully (retried 5 times)." application=myapp-controlplane-configs-dev dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" reason=OperationCompleted type=Warning
time="2023-09-21T19:10:28Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:10:28Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: api-argocd-dev)" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:10:28Z" level=info msg="Refreshing app status (controller refresh requested), level (2)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:28Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:28Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/bootstrap-cluster-api-platform-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=8 unmarshal_ms=8 version_ms=0
time="2023-09-21T19:10:28Z" level=info msg="Skipping auto-sync: application status is Synced" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:10:28Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/bootstrap-cluster-api-platform-dev
time="2023-09-21T19:10:28Z" level=info msg="Reconciliation completed" application=api-argocd-dev/bootstrap-cluster-api-platform-dev dedup_ms=0 dest-name= dest-namespace=api-argocd-dev dest-server="https://kubernetes.default.svc" diff_ms=18 fields.level=1 git_ms=8 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=44
t...
time="2023-09-21T19:10:54Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=10 unmarshal_ms=9 version_ms=0
time="2023-09-21T19:10:54Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=8 unmarshal_ms=8 version_ms=0
time="2023-09-21T19:10:54Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00217-lSCUg
time="2023-09-21T19:10:55Z" level=info msg="Skipping auto-sync: another operation is in progress" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dedup_ms=0 dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" diff_ms=405 fields.level=1 git_ms=10 health_ms=1 live_ms=0 settings_ms=0 sync_ms=0 time_ms=619
time="2023-09-21T19:10:55Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00217-lSCUg tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,serviceaccount/certificate-exporter unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (Synced,Succeeded,configmap/myapp-controlplane-configs-checksums configured), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,configmap/certificate-exporter unchanged), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,Running,job.batch/myjob created), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,cronjob.batch/certificate-exporter unchanged)]"
time="2023-09-21T19:10:55Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=203.975419ms syncId=00217-lSCUg
time="2023-09-21T19:10:55Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=8 unmarshal_ms=8 version_ms=0
time="2023-09-21T19:10:55Z" level=info msg="No operation updates necessary to 'api-argocd-dev/myapp-controlplane-configs-dev'. Skipping patch" appNamespace=api-argocd-dev application=myapp-controlplane-configs-dev project=myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="Resuming in-progress operation. phase: Running, message: waiting for completion of hook batch/Job/myjob" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: myapp-controlplane-dev)" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="getRepoObjs stats" application=api-argocd-dev/myapp-controlplane-configs-dev build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=7 unmarshal_ms=7 version_ms=0
time="2023-09-21T19:10:55Z" level=info msg="Skipping auto-sync: another operation is in progress" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="No status changes. Skipping patch" application=api-argocd-dev/myapp-controlplane-configs-dev
time="2023-09-21T19:10:55Z" level=info msg="Reconciliation completed" application=api-argocd-dev/myapp-controlplane-configs-dev dedup_ms=0 dest-name= dest-namespace=myapp-controlplane-dev dest-server="https://kubernetes.default.svc" diff_ms=13 fields.level=1 git_ms=8 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=34
time="2023-09-21T19:10:55Z" level=info msg=Syncing application=api-argocd-dev/myapp-controlplane-configs-dev skipHooks=false started=true syncId=00218-lBylj
time="2023-09-21T19:10:55Z" level=info msg=Tasks application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00218-lBylj tasks="[Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/secret obj->obj (Synced,Succeeded,Secret was synced), Sync/-5 resource external-secrets.io/ExternalSecret:myapp-controlplane-dev/git-access-token obj->obj (Synced,Succeeded,Secret was synced), Sync/0 resource /ServiceAccount:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,serviceaccount/certificate-exporter unchanged), Sync/0 resource /ConfigMap:myapp-controlplane-dev/myapp-controlplane-configs-checksums obj->obj (Synced,Succeeded,configmap/myapp-controlplane-configs-checksums configured), Sync/0 resource /ConfigMap:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,configmap/certificate-exporter unchanged), Sync/0 hook batch/Job:myapp-controlplane-dev/myjob obj->obj (,Running,job.batch/myjob created), Sync/0 resource batch/CronJob:myapp-controlplane-dev/certificate-exporter obj->obj (Synced,Succeeded,cronjob.batch/certificate-exporter unchanged)]"
time="2023-09-21T19:10:55Z" level=info msg="Updating resource result, status: '' -> '', phase 'Running' -> 'Succeeded', message 'job.batch/myjob created' -> 'job.batch/myjob created'" application=api-argocd-dev/myapp-controlplane-configs-dev kind=Job name=myjob namespace=myapp-controlplane-dev phase=Sync syncId=00218-lBylj
time="2023-09-21T19:10:55Z" level=info msg="Updating operation state. phase: Running -> Succeeded, message: 'waiting for completion of hook batch/Job/myjob' -> 'successfully synced (no more tasks)'" application=api-argocd-dev/myapp-controlplane-configs-dev syncId=00218-lBylj
time="2023-09-21T19:10:55Z" level=info msg="sync/terminate complete" application=api-argocd-dev/myapp-controlplane-configs-dev duration=8.793144ms syncId=00218-lBylj

[fabry00]

if I set in the Application yaml

retry: 
  limit: 1
  backoff:
    duration: 1s
    factor: 1
    maxDuration: 1m

Then ArgoCD retries 2 times and then it fails. Not perfect but better than waiting 10 minutes. But If I do not set the rety, ArgoCD UI shows: Retry Disabled so I think it's a bug because it retries 5 times

[crenshaw-dev]

I'm having trouble running down the code, but to me it looks like an unchecked "Retry" option in the UI just means "use the default or app-configured retry config." It doesn't seem like the unchecked box means "don't retry."

[fabry00]

It doesn't seem like the unchecked box means "don't retry."

Not sure I understood, but from the UI you can clearly see "Retry Disabled" in my screenshot

[jannfis]

Yes, I believe there is a discrepancy between the UI and the default behavior here.

The default is to retry 5 times, but the UI assumes it to be disabled when not configured. Potentially, the default has been changed from "Disabled if not specified" to "Retry 5 times if not specified". And it seems only to apply to autosync: https://github.com/argoproj/argo-cd/blob/28ef0961b34f098eb6b7631ecc65f0a3a42ff85f/controller/appcontroller.go#L1723-L1735

So, FWIW, this is a bug. For autosync, you can't seem to disable retries. If you don't set the retries, the default will be assumed, if you do set the retries, it will retry at least once (because 0 means unlimited). This is not solvable in an elegant way without breaking existing behaviour (people may rely on it), so what comes to my mind is to give the retry stanza a enabled (or disabled) field, so you could do something like:

retry:
  enabled: false