Server-side diff fails when syncing ExternalSecret resource

[karlschriek]

Checklist:

• [x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [x] I've included steps to reproduce the bug.
• [x] I've pasted the output of argocd version.

Describe the bug

Whenever ArgoCD attempts to sync an ExternalSecret resource, we get errors such as these:

Failed to compare desired state to live state: failed to calculate diff from cache: error calculating server side diff: serverSideDiff error: error running server side apply in dryrun mode for resource ExternalSecret/mysecret: ExternalSecret.external-secrets.io "mysecret" is invalid: [spec.data[0].remoteRef: Required value, spec.data[1].remoteRef: Required value, spec.data[2].remoteRef: Required value, spec.data[3].remoteRef: Required value]

And the app goes into a sync status of "Unknown". The strange thing is if we manually do a sync via the UI, the status will resolve to "synced", but a few minutes later the same error will occur and status will go back to "Unknown".

To Reproduce

With an ArgoCD distro deployed with controller.diff.server.side: "true", create an Application that deploys an ExternalSecret. After a while, the above error will start popping up.

Expected behavior

The sync should succeed.

Screenshots

Version

2.11.4

Logs

Paste any relevant application logs here.

[christianh814]

Do you know if this is for a specific ESO provider or does it happen with all providers?

[ChristianCiach]

Are you using ignoreDifferences? Looks related to

• https://github.com/argoproj/argo-cd/issues/17362

[karlschriek]

I have not noticed it on any resources other than ExternalSecrets.

There are some global "ignoreDifferences" set, yes, in particular this one, which I presume might play a role here:

  resource.customizations.ignoreDifferences.external-secrets.io_ExternalSecret: |
      jqPathExpressions:
        - '.spec.data[]?.remoteRef'