search

argoproj / argo-cd

Declarative Continuous Deployment for Kubernetes
https://argo-cd.readthedocs.io
Apache License 2.0
18k stars 5.48k forks source link

successfull sync doesn't sync #20875

Open stephaneetje opened 1 day ago

stephaneetje commented 1 day ago

I'm using kargo to update the image tag. When argocd syncs from kargo, i can see it uses the right commit sha with my image tag update. Sync ends successfully, but deployment hasn't been updated and app is still out of sync untill i sync manually.

Argocd version: 2.13

Capture d’écran 2024-11-20 à 23 18 59 Capture d’écran 2024-11-20 à 23 19 30 Capture d’écran 2024-11-20 à 23 21 43
andrii-korotkov-verkada commented 1 day ago

Does it show as out of sync? I saw a similar issue recently. It shows out of sync and not applying all changes in a previous sync, but manual sync can get it to a good state.

andrii-korotkov-verkada commented 1 day ago

Can you try to get logs with debug enabled, please?

Try setting this

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cmd-params-cm
data:
  applicationsetcontroller.log.level: debug
  applicationsetcontroller.log.format: json
  controller.log.level: debug
  controller.log.format: json
  notificationscontroller.log.level: debug
  notificationscontroller.log.format: json
  reposerver.log.level: debug
  reposerver.log.format: json
  server.log.level: debug
  server.log.format: json
  ...
stephaneetje commented 1 day ago

applicationsetcontroller.log.level: debug applicationsetcontroller.log.format: json controller.log.level: debug controller.log.format: json notificationscontroller.log.level: debug notificationscontroller.log.format: json reposerver.log.level: debug reposerver.log.format: json server.log.level: debug server.log.format: json

Here are the application-controller logs:

{"level":"debug","msg":"Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0","time":"2024-11-21T08:46:19Z"}
{"api-version":"argoproj.io/v1alpha1","app-namespace":"argocd","app-qualified-name":"argocd/apps","application":"apps","cluster-name":"","comparison-level":0,"kind":"Application","level":"debug","msg":"Requesting app refresh caused by object update","name":"scim-staging","namespace":"argocd","project":"default","server":"https://kubernetes.default.svc","time":"2024-11-21T08:46:19Z"}
{"level":"debug","msg":"Skipping sharding distribution update. No relevant changes","time":"2024-11-21T08:46:19Z"}
{"app-namespace":"argocd","app-qualified-name":"argocd/apps","application":"apps","level":"info","msg":"Refreshing app status (controller refresh requested), level (0)","project":"default","time":"2024-11-21T08:46:19Z"}
{"level":"info","msg":"Completed Update application operation state","time":"2024-11-21T08:46:19Z"}
{"app-namespace":"argocd","app-qualified-name":"argocd/scim-staging","application":"scim-staging","level":"info","msg":"updated 'argocd/scim-staging' operation (phase: Succeeded)","project":"scim-staging","time":"2024-11-21T08:46:19Z"}
{"application":"scim-staging","dest-namespace":"scim-staging","dest-server":"https://35.25.15.168 », »level":"info","msg":"Sync operation to b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e succeeded","reason":"OperationCompleted","time":"2024-11-21T08:46:19Z","type":"Normal"}
{"app-namespace":"argocd","app-qualified-name":"argocd/apps","application":"apps","iterate_resources_ms":1,"level":"debug","msg":"Finished getting app hosts","process_app_pods_by_node_ms":0,"project":"default","time":"2024-11-21T08:46:19Z","time_ms":1}
{"app-namespace":"argocd","app-qualified-name":"argocd/apps","application":"apps","get_app_hosts_ms":1,"get_app_proj_ms":0,"get_orphaned_resources_ms":0,"level":"debug","msg":"Finished getting resource tree","process_managed_resources_ms":22,"process_orphaned_resources_ms":0,"project":"default","time":"2024-11-21T08:46:19Z","time_ms":23}
{"app-namespace":"argocd","app-qualified-name":"argocd/scim-staging","application":"scim-staging","final_set_operation_state":105,"get_app_proj_ms":0,"initial_operation_stage_ms":0,"level":"debug","msg":"Finished processing requested app operation","project":"scim-staging","request_app_refresh_ms":0,"time":"2024-11-21T08:46:19Z","time_ms":265,"validate_and_sync_app_state_ms":159}
{"app-namespace":"argocd","app-qualified-name":"argocd/scim-staging","application":"scim-staging","get_fresh_app_ms":23,"level":"debug","msg":"Finished processing app operation queue item","process_requested_app_operation_ms":265,"project":"scim-staging","time":"2024-11-21T08:46:19Z","time_ms":289}
{"app-namespace":"argocd","app-qualified-name":"argocd/scim-staging","application":"scim-staging","get_fresh_app_ms":0,"level":"debug","msg":"Finished processing app operation queue item","project":"scim-staging","time":"2024-11-21T08:46:19Z","time_ms":0}
{"app-namespace":"argocd","app-qualified-name":"argocd/apps","application":"apps","level":"info","msg":"No status changes. Skipping patch","project":"default","time":"2024-11-21T08:46:19Z"}
{"app-namespace":"argocd","app-qualified-name":"argocd/apps","application":"apps","comparison-level":0,"dest-name":"","dest-namespace":"argocd","dest-server":"https://kubernetes.default.svc","level":"info","msg":"Reconciliation completed","patch_ms":0,"project":"default","setop_ms":0,"time":"2024-11-21T08:46:19Z","time_ms":33}
{"app-namespace":"argocd","app-qualified-name":"argocd/scim-staging","application":"scim-staging","level":"info","msg":"Refreshing app status (controller refresh requested), level (3)","project":"scim-staging","time":"2024-11-21T08:46:19Z"}
{"application":"argocd/scim-staging","level":"info","msg":"Comparing app state (cluster: https://35.25.15.168, namespace: scim-staging)","time":"2024-11-21T08:46:19Z"}
{"level":"debug","msg":"https://github.com/test/ciam-helm has credentials","time":"2024-11-21T08:46:19Z"}
{"level":"debug","msg":"Generating Manifest for source {https://github.com/test/ciam-helm helm/scim main \u0026ApplicationSourceHelm{ValueFiles:[secrets://values-scim-staging.yaml values-scim-staging-image.yaml],Parameters:[]HelmParameter{},ReleaseName:,Values:,FileParameters:[]HelmFileParameter{},Version:,PassCredentials:false,IgnoreMissingValueFiles:false,SkipCrds:false,ValuesObject:nil,Namespace:,KubeVersion:,APIVersions:[],} nil nil nil  } revision main","time":"2024-11-21T08:46:19Z"}

repo-server:

{"appName":"scim-staging","appSrc":"{\"appSrc\":{\"repoURL\":\"\",\"path\":\"helm/scim\",\"helm\":{\"valueFiles\":[\"secrets://values-scim-staging.yaml\",\"values-scim-staging-image.yaml\"]}},\"srcRefs\":{}}","clusterInfo":"1.30|acme.cert-manager.io/v1,acme.cert-manager.io/v1/Challenge,acme.cert-manager.io/v1/Order,admissionregistration.k8s.io/v1,admissionregistration.k8s.io/v1/MutatingWebhookConfiguration,admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy,admissionregistration.k8s.io/v1/ValidatingAdmissionPolicyBinding,admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration,apiextensions.k8s.io/v1,apiextensions.k8s.io/v1/CustomResourceDefinition,apiregistration.k8s.io/v1,apiregistration.k8s.io/v1/APIService,apps/v1,apps/v1/ControllerRevision,apps/v1/DaemonSet,apps/v1/Deployment,apps/v1/ReplicaSet,apps/v1/StatefulSet,auto.gke.io/v1,auto.gke.io/v1/AllowlistedV2Workload,auto.gke.io/v1/AllowlistedWorkload,auto.gke.io/v1/WorkloadAllowlist,auto.gke.io/v1alpha1,auto.gke.io/v1alpha1/AllowlistedWorkload,autoscaling.gke.io/v1beta1,autoscaling.gke.io/v1beta1/MultidimPodAutoscaler,autoscaling.k8s.io/v1,autoscaling.k8s.io/v1/VerticalPodAutoscaler,autoscaling.k8s.io/v1/VerticalPodAutoscalerCheckpoint,autoscaling.k8s.io/v1beta2,autoscaling.k8s.io/v1beta2/VerticalPodAutoscaler,autoscaling.k8s.io/v1beta2/VerticalPodAutoscalerCheckpoint,autoscaling.x-k8s.io/v1beta1,autoscaling.x-k8s.io/v1beta1/ProvisioningRequest,autoscaling/v1,autoscaling/v1/HorizontalPodAutoscaler,autoscaling/v2,autoscaling/v2/HorizontalPodAutoscaler,batch/v1,batch/v1/CronJob,batch/v1/Job,cert-manager.io/v1,cert-manager.io/v1/Certificate,cert-manager.io/v1/CertificateRequest,cert-manager.io/v1/ClusterIssuer,cert-manager.io/v1/Issuer,certificates.k8s.io/v1,certificates.k8s.io/v1/CertificateSigningRequest,cloud.google.com/v1,cloud.google.com/v1/BackendConfig,cloud.google.com/v1/ComputeClass,cloud.google.com/v1beta1,cloud.google.com/v1beta1/BackendConfig,coordination.k8s.io/v1,coordination.k8s.io/v1/Lease,crd.projectcalico.org/v1,crd.projectcalico.org/v1/BGPConfiguration,crd.projectcalico.org/v1/BGPFilter,crd.projectcalico.org/v1/BGPPeer,crd.projectcalico.org/v1/BlockAffinity,crd.projectcalico.org/v1/CalicoNodeStatus,crd.projectcalico.org/v1/ClusterInformation,crd.projectcalico.org/v1/FelixConfiguration,crd.projectcalico.org/v1/GlobalNetworkPolicy,crd.projectcalico.org/v1/GlobalNetworkSet,crd.projectcalico.org/v1/HostEndpoint,crd.projectcalico.org/v1/IPAMBlock,crd.projectcalico.org/v1/IPAMConfig,crd.projectcalico.org/v1/IPAMHandle,crd.projectcalico.org/v1/IPPool,crd.projectcalico.org/v1/IPReservation,crd.projectcalico.org/v1/KubeControllersConfiguration,crd.projectcalico.org/v1/NetworkPolicy,crd.projectcalico.org/v1/NetworkSet,discovery.k8s.io/v1,discovery.k8s.io/v1/EndpointSlice,events.k8s.io/v1,events.k8s.io/v1/Event,flowcontrol.apiserver.k8s.io/v1,flowcontrol.apiserver.k8s.io/v1/FlowSchema,flowcontrol.apiserver.k8s.io/v1/PriorityLevelConfiguration,flowcontrol.apiserver.k8s.io/v1beta3,flowcontrol.apiserver.k8s.io/v1beta3/FlowSchema,flowcontrol.apiserver.k8s.io/v1beta3/PriorityLevelConfiguration,hub.gke.io/v1,hub.gke.io/v1/Membership,internal.autoscaling.gke.io/v1,internal.autoscaling.gke.io/v1/CapacityRequest,kubecost.com/v1alpha1,kubecost.com/v1alpha1/TurndownSchedule,networking.gke.io/v1,networking.gke.io/v1/GKENetworkParamSet,networking.gke.io/v1/ManagedCertificate,networking.gke.io/v1/Network,networking.gke.io/v1/ServiceAttachment,networking.gke.io/v1beta1,networking.gke.io/v1beta1/FrontendConfig,networking.gke.io/v1beta1/ManagedCertificate,networking.gke.io/v1beta1/ServiceAttachment,networking.gke.io/v1beta1/ServiceNetworkEndpointGroup,networking.gke.io/v1beta2,networking.gke.io/v1beta2/ManagedCertificate,networking.k8s.io/v1,networking.k8s.io/v1/Ingress,networking.k8s.io/v1/IngressClass,networking.k8s.io/v1/NetworkPolicy,node.gke.io/v1,node.gke.io/v1/GCPResourceAllowlist,node.k8s.io/v1,node.k8s.io/v1/RuntimeClass,nodemanagement.gke.io/v1alpha1,nodemanagement.gke.io/v1alpha1/UpdateInfo,policy/v1,policy/v1/PodDisruptionBudget,rbac.authorization.k8s.io/v1,rbac.authorization.k8s.io/v1/ClusterRole,rbac.authorization.k8s.io/v1/ClusterRoleBinding,rbac.authorization.k8s.io/v1/Role,rbac.authorization.k8s.io/v1/RoleBinding,scheduling.k8s.io/v1,scheduling.k8s.io/v1/PriorityClass,snapshot.storage.k8s.io/v1,snapshot.storage.k8s.io/v1/VolumeSnapshot,snapshot.storage.k8s.io/v1/VolumeSnapshotClass,snapshot.storage.k8s.io/v1/VolumeSnapshotContent,snapshot.storage.k8s.io/v1beta1,snapshot.storage.k8s.io/v1beta1/VolumeSnapshot,snapshot.storage.k8s.io/v1beta1/VolumeSnapshotClass,snapshot.storage.k8s.io/v1beta1/VolumeSnapshotContent,storage.k8s.io/v1,storage.k8s.io/v1/CSIDriver,storage.k8s.io/v1/CSINode,storage.k8s.io/v1/CSIStorageCapacity,storage.k8s.io/v1/StorageClass,storage.k8s.io/v1/VolumeAttachment,v1,v1/ConfigMap,v1/Endpoints,v1/Event,v1/LimitRange,v1/Namespace,v1/Node,v1/PersistentVolume,v1/PersistentVolumeClaim,v1/Pod,v1/PodTemplate,v1/ReplicationController,v1/ResourceQuota,v1/Secret,v1/Service,v1/ServiceAccount,warden.gke.io/v1,warden.gke.io/v1/Audit","level":"debug","msg":"getting manifests cache","namespace":"scim-staging","reason":"GenerateManifest API call","revision":"b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e","time":"2024-11-21T08:46:17Z","trackingKey":"annotation:argocd.argoproj.io/instance"}
{"level":"info","msg":"manifest cache hit: \u0026ApplicationSource{RepoURL:https://github.com/test/ciam-helm,Path:helm/scim,TargetRevision:main,Helm:\u0026ApplicationSourceHelm{ValueFiles:[secrets://values-scim-staging.yaml values-scim-staging-image.yaml],Parameters:[]HelmParameter{},ReleaseName:,Values:,FileParameters:[]HelmFileParameter{},Version:,PassCredentials:false,IgnoreMissingValueFiles:false,SkipCrds:false,ValuesObject:nil,Namespace:,KubeVersion:,APIVersions:[],},Kustomize:nil,Directory:nil,Plugin:nil,Chart:,Ref:,}/b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e","time":"2024-11-21T08:46:17Z"}
{"appName":"scim-staging","appSrc":"{\"appSrc\":{\"repoURL\":\"\",\"path\":\"helm/scim\",\"helm\":{\"valueFiles\":[\"secrets://values-scim-staging.yaml\",\"values-scim-staging-image.yaml\"]}},\"srcRefs\":{}}","clusterInfo":"1.30|acme.cert-manager.io/v1,acme.cert-manager.io/v1/Challenge,acme.cert-manager.io/v1/Order,admissionregistration.k8s.io/v1,admissionregistration.k8s.io/v1/MutatingWebhookConfiguration,admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy,admissionregistration.k8s.io/v1/ValidatingAdmissionPolicyBinding,admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration,apiextensions.k8s.io/v1,apiextensions.k8s.io/v1/CustomResourceDefinition,apiregistration.k8s.io/v1,apiregistration.k8s.io/v1/APIService,apps/v1,apps/v1/ControllerRevision,apps/v1/DaemonSet,apps/v1/Deployment,apps/v1/ReplicaSet,apps/v1/StatefulSet,auto.gke.io/v1,auto.gke.io/v1/AllowlistedV2Workload,auto.gke.io/v1/AllowlistedWorkload,auto.gke.io/v1/WorkloadAllowlist,auto.gke.io/v1alpha1,auto.gke.io/v1alpha1/AllowlistedWorkload,autoscaling.gke.io/v1beta1,autoscaling.gke.io/v1beta1/MultidimPodAutoscaler,autoscaling.k8s.io/v1,autoscaling.k8s.io/v1/VerticalPodAutoscaler,autoscaling.k8s.io/v1/VerticalPodAutoscalerCheckpoint,autoscaling.k8s.io/v1beta2,autoscaling.k8s.io/v1beta2/VerticalPodAutoscaler,autoscaling.k8s.io/v1beta2/VerticalPodAutoscalerCheckpoint,autoscaling.x-k8s.io/v1beta1,autoscaling.x-k8s.io/v1beta1/ProvisioningRequest,autoscaling/v1,autoscaling/v1/HorizontalPodAutoscaler,autoscaling/v2,autoscaling/v2/HorizontalPodAutoscaler,batch/v1,batch/v1/CronJob,batch/v1/Job,cert-manager.io/v1,cert-manager.io/v1/Certificate,cert-manager.io/v1/CertificateRequest,cert-manager.io/v1/ClusterIssuer,cert-manager.io/v1/Issuer,certificates.k8s.io/v1,certificates.k8s.io/v1/CertificateSigningRequest,cloud.google.com/v1,cloud.google.com/v1/BackendConfig,cloud.google.com/v1/ComputeClass,cloud.google.com/v1beta1,cloud.google.com/v1beta1/BackendConfig,coordination.k8s.io/v1,coordination.k8s.io/v1/Lease,crd.projectcalico.org/v1,crd.projectcalico.org/v1/BGPConfiguration,crd.projectcalico.org/v1/BGPFilter,crd.projectcalico.org/v1/BGPPeer,crd.projectcalico.org/v1/BlockAffinity,crd.projectcalico.org/v1/CalicoNodeStatus,crd.projectcalico.org/v1/ClusterInformation,crd.projectcalico.org/v1/FelixConfiguration,crd.projectcalico.org/v1/GlobalNetworkPolicy,crd.projectcalico.org/v1/GlobalNetworkSet,crd.projectcalico.org/v1/HostEndpoint,crd.projectcalico.org/v1/IPAMBlock,crd.projectcalico.org/v1/IPAMConfig,crd.projectcalico.org/v1/IPAMHandle,crd.projectcalico.org/v1/IPPool,crd.projectcalico.org/v1/IPReservation,crd.projectcalico.org/v1/KubeControllersConfiguration,crd.projectcalico.org/v1/NetworkPolicy,crd.projectcalico.org/v1/NetworkSet,discovery.k8s.io/v1,discovery.k8s.io/v1/EndpointSlice,events.k8s.io/v1,events.k8s.io/v1/Event,flowcontrol.apiserver.k8s.io/v1,flowcontrol.apiserver.k8s.io/v1/FlowSchema,flowcontrol.apiserver.k8s.io/v1/PriorityLevelConfiguration,flowcontrol.apiserver.k8s.io/v1beta3,flowcontrol.apiserver.k8s.io/v1beta3/FlowSchema,flowcontrol.apiserver.k8s.io/v1beta3/PriorityLevelConfiguration,hub.gke.io/v1,hub.gke.io/v1/Membership,internal.autoscaling.gke.io/v1,internal.autoscaling.gke.io/v1/CapacityRequest,kubecost.com/v1alpha1,kubecost.com/v1alpha1/TurndownSchedule,networking.gke.io/v1,networking.gke.io/v1/GKENetworkParamSet,networking.gke.io/v1/ManagedCertificate,networking.gke.io/v1/Network,networking.gke.io/v1/ServiceAttachment,networking.gke.io/v1beta1,networking.gke.io/v1beta1/FrontendConfig,networking.gke.io/v1beta1/ManagedCertificate,networking.gke.io/v1beta1/ServiceAttachment,networking.gke.io/v1beta1/ServiceNetworkEndpointGroup,networking.gke.io/v1beta2,networking.gke.io/v1beta2/ManagedCertificate,networking.k8s.io/v1,networking.k8s.io/v1/Ingress,networking.k8s.io/v1/IngressClass,networking.k8s.io/v1/NetworkPolicy,node.gke.io/v1,node.gke.io/v1/GCPResourceAllowlist,node.k8s.io/v1,node.k8s.io/v1/RuntimeClass,nodemanagement.gke.io/v1alpha1,nodemanagement.gke.io/v1alpha1/UpdateInfo,policy/v1,policy/v1/PodDisruptionBudget,rbac.authorization.k8s.io/v1,rbac.authorization.k8s.io/v1/ClusterRole,rbac.authorization.k8s.io/v1/ClusterRoleBinding,rbac.authorization.k8s.io/v1/Role,rbac.authorization.k8s.io/v1/RoleBinding,scheduling.k8s.io/v1,scheduling.k8s.io/v1/PriorityClass,snapshot.storage.k8s.io/v1,snapshot.storage.k8s.io/v1/VolumeSnapshot,snapshot.storage.k8s.io/v1/VolumeSnapshotClass,snapshot.storage.k8s.io/v1/VolumeSnapshotContent,snapshot.storage.k8s.io/v1beta1,snapshot.storage.k8s.io/v1beta1/VolumeSnapshot,snapshot.storage.k8s.io/v1beta1/VolumeSnapshotClass,snapshot.storage.k8s.io/v1beta1/VolumeSnapshotContent,storage.k8s.io/v1,storage.k8s.io/v1/CSIDriver,storage.k8s.io/v1/CSINode,storage.k8s.io/v1/CSIStorageCapacity,storage.k8s.io/v1/StorageClass,storage.k8s.io/v1/VolumeAttachment,v1,v1/ConfigMap,v1/Endpoints,v1/Event,v1/LimitRange,v1/Namespace,v1/Node,v1/PersistentVolume,v1/PersistentVolumeClaim,v1/Pod,v1/PodTemplate,v1/ReplicationController,v1/ResourceQuota,v1/Secret,v1/Service,v1/ServiceAccount,warden.gke.io/v1,warden.gke.io/v1/Audit","level":"debug","msg":"getting manifests cache","namespace":"scim-staging","reason":"GenerateManifest API call","revision":"b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e","time":"2024-11-21T08:46:19Z","trackingKey":"annotation:argocd.argoproj.io/instance"}
{"level":"info","msg":"manifest cache hit: \u0026ApplicationSource{RepoURL:https://github.com/test/ciam-helm,Path:helm/scim,TargetRevision:main,Helm:\u0026ApplicationSourceHelm{ValueFiles:[secrets://values-scim-staging.yaml values-scim-staging-image.yaml],Parameters:[]HelmParameter{},ReleaseName:,Values:,FileParameters:[]HelmFileParameter{},Version:,PassCredentials:false,IgnoreMissingValueFiles:false,SkipCrds:false,ValuesObject:nil,Namespace:,KubeVersion:,APIVersions:[],},Kustomize:nil,Directory:nil,Plugin:nil,Chart:,Ref:,}/b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e","time":"2024-11-21T08:46:19Z"}
{"level":"debug","msg":"revision 'main' resolved to 'b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e'","time":"2024-11-21T08:46:20Z"}
{"appName":"scim-staging","appSrc":"{\"appSrc\":{\"repoURL\":\"\",\"path\":\"helm/scim\",\"helm\":{\"valueFiles\":[\"secrets://values-scim-staging.yaml\",\"values-scim-staging-image.yaml\"]}},\"srcRefs\":{}}","clusterInfo":"1.30|acme.cert-manager.io/v1,acme.cert-manager.io/v1/Challenge,acme.cert-manager.io/v1/Order,admissionregistration.k8s.io/v1,admissionregistration.k8s.io/v1/MutatingWebhookConfiguration,admissionregistration.k8s.io/v1/ValidatingAdmissionPolicy,admissionregistration.k8s.io/v1/ValidatingAdmissionPolicyBinding,admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration,apiextensions.k8s.io/v1,apiextensions.k8s.io/v1/CustomResourceDefinition,apiregistration.k8s.io/v1,apiregistration.k8s.io/v1/APIService,apps/v1,apps/v1/ControllerRevision,apps/v1/DaemonSet,apps/v1/Deployment,apps/v1/ReplicaSet,apps/v1/StatefulSet,auto.gke.io/v1,auto.gke.io/v1/AllowlistedV2Workload,auto.gke.io/v1/AllowlistedWorkload,auto.gke.io/v1/WorkloadAllowlist,auto.gke.io/v1alpha1,auto.gke.io/v1alpha1/AllowlistedWorkload,autoscaling.gke.io/v1beta1,autoscaling.gke.io/v1beta1/MultidimPodAutoscaler,autoscaling.k8s.io/v1,autoscaling.k8s.io/v1/VerticalPodAutoscaler,autoscaling.k8s.io/v1/VerticalPodAutoscalerCheckpoint,autoscaling.k8s.io/v1beta2,autoscaling.k8s.io/v1beta2/VerticalPodAutoscaler,autoscaling.k8s.io/v1beta2/VerticalPodAutoscalerCheckpoint,autoscaling.x-k8s.io/v1beta1,autoscaling.x-k8s.io/v1beta1/ProvisioningRequest,autoscaling/v1,autoscaling/v1/HorizontalPodAutoscaler,autoscaling/v2,autoscaling/v2/HorizontalPodAutoscaler,batch/v1,batch/v1/CronJob,batch/v1/Job,cert-manager.io/v1,cert-manager.io/v1/Certificate,cert-manager.io/v1/CertificateRequest,cert-manager.io/v1/ClusterIssuer,cert-manager.io/v1/Issuer,certificates.k8s.io/v1,certificates.k8s.io/v1/CertificateSigningRequest,cloud.google.com/v1,cloud.google.com/v1/BackendConfig,cloud.google.com/v1/ComputeClass,cloud.google.com/v1beta1,cloud.google.com/v1beta1/BackendConfig,coordination.k8s.io/v1,coordination.k8s.io/v1/Lease,crd.projectcalico.org/v1,crd.projectcalico.org/v1/BGPConfiguration,crd.projectcalico.org/v1/BGPFilter,crd.projectcalico.org/v1/BGPPeer,crd.projectcalico.org/v1/BlockAffinity,crd.projectcalico.org/v1/CalicoNodeStatus,crd.projectcalico.org/v1/ClusterInformation,crd.projectcalico.org/v1/FelixConfiguration,crd.projectcalico.org/v1/GlobalNetworkPolicy,crd.projectcalico.org/v1/GlobalNetworkSet,crd.projectcalico.org/v1/HostEndpoint,crd.projectcalico.org/v1/IPAMBlock,crd.projectcalico.org/v1/IPAMConfig,crd.projectcalico.org/v1/IPAMHandle,crd.projectcalico.org/v1/IPPool,crd.projectcalico.org/v1/IPReservation,crd.projectcalico.org/v1/KubeControllersConfiguration,crd.projectcalico.org/v1/NetworkPolicy,crd.projectcalico.org/v1/NetworkSet,discovery.k8s.io/v1,discovery.k8s.io/v1/EndpointSlice,events.k8s.io/v1,events.k8s.io/v1/Event,flowcontrol.apiserver.k8s.io/v1,flowcontrol.apiserver.k8s.io/v1/FlowSchema,flowcontrol.apiserver.k8s.io/v1/PriorityLevelConfiguration,flowcontrol.apiserver.k8s.io/v1beta3,flowcontrol.apiserver.k8s.io/v1beta3/FlowSchema,flowcontrol.apiserver.k8s.io/v1beta3/PriorityLevelConfiguration,hub.gke.io/v1,hub.gke.io/v1/Membership,internal.autoscaling.gke.io/v1,internal.autoscaling.gke.io/v1/CapacityRequest,kubecost.com/v1alpha1,kubecost.com/v1alpha1/TurndownSchedule,networking.gke.io/v1,networking.gke.io/v1/GKENetworkParamSet,networking.gke.io/v1/ManagedCertificate,networking.gke.io/v1/Network,networking.gke.io/v1/ServiceAttachment,networking.gke.io/v1beta1,networking.gke.io/v1beta1/FrontendConfig,networking.gke.io/v1beta1/ManagedCertificate,networking.gke.io/v1beta1/ServiceAttachment,networking.gke.io/v1beta1/ServiceNetworkEndpointGroup,networking.gke.io/v1beta2,networking.gke.io/v1beta2/ManagedCertificate,networking.k8s.io/v1,networking.k8s.io/v1/Ingress,networking.k8s.io/v1/IngressClass,networking.k8s.io/v1/NetworkPolicy,node.gke.io/v1,node.gke.io/v1/GCPResourceAllowlist,node.k8s.io/v1,node.k8s.io/v1/RuntimeClass,nodemanagement.gke.io/v1alpha1,nodemanagement.gke.io/v1alpha1/UpdateInfo,policy/v1,policy/v1/PodDisruptionBudget,rbac.authorization.k8s.io/v1,rbac.authorization.k8s.io/v1/ClusterRole,rbac.authorization.k8s.io/v1/ClusterRoleBinding,rbac.authorization.k8s.io/v1/Role,rbac.authorization.k8s.io/v1/RoleBinding,scheduling.k8s.io/v1,scheduling.k8s.io/v1/PriorityClass,snapshot.storage.k8s.io/v1,snapshot.storage.k8s.io/v1/VolumeSnapshot,snapshot.storage.k8s.io/v1/VolumeSnapshotClass,snapshot.storage.k8s.io/v1/VolumeSnapshotContent,snapshot.storage.k8s.io/v1beta1,snapshot.storage.k8s.io/v1beta1/VolumeSnapshot,snapshot.storage.k8s.io/v1beta1/VolumeSnapshotClass,snapshot.storage.k8s.io/v1beta1/VolumeSnapshotContent,storage.k8s.io/v1,storage.k8s.io/v1/CSIDriver,storage.k8s.io/v1/CSINode,storage.k8s.io/v1/CSIStorageCapacity,storage.k8s.io/v1/StorageClass,storage.k8s.io/v1/VolumeAttachment,v1,v1/ConfigMap,v1/Endpoints,v1/Event,v1/LimitRange,v1/Namespace,v1/Node,v1/PersistentVolume,v1/PersistentVolumeClaim,v1/Pod,v1/PodTemplate,v1/ReplicationController,v1/ResourceQuota,v1/Secret,v1/Service,v1/ServiceAccount,warden.gke.io/v1,warden.gke.io/v1/Audit","level":"debug","msg":"getting manifests cache","namespace":"scim-staging","reason":"GenerateManifest API call","revision":"b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e","time":"2024-11-21T08:46:20Z","trackingKey":"annotation:argocd.argoproj.io/instance"}
{"level":"info","msg":"manifest cache hit: \u0026ApplicationSource{RepoURL:https://github.com/test/ciam-helm,Path:helm/scim,TargetRevision:main,Helm:\u0026ApplicationSourceHelm{ValueFiles:[secrets://values-scim-staging.yaml values-scim-staging-image.yaml],Parameters:[]HelmParameter{},ReleaseName:,Values:,FileParameters:[]HelmFileParameter{},Version:,PassCredentials:false,IgnoreMissingValueFiles:false,SkipCrds:false,ValuesObject:nil,Namespace:,KubeVersion:,APIVersions:[],},Kustomize:nil,Directory:nil,Plugin:nil,Chart:,Ref:,}/b7d7a7cac348aac80e1fdd5474d0b2eb0334a01e","time":"2024-11-21T08:46:20Z"}

I had to run it multiple times to catch the logs as there are many replicas + lot of applications flooding logs.

sunilhonest commented 4 hours ago

I am facing the same issue. I have progressive syncs enabled in ArgoCD applicationset and the application status is OutofSync despite it syncing successfully. I am able to see the diff in the image tag of deployment that doesn't get reconciled.

@stephaneetje Do you have progressive syncs enabled ?

stephaneetje commented 3 hours ago

@stephaneetje Do you have progressive syncs enabled ?

Hello, no I never activated progressive syncs.