Argo CD validating duplicate variables and causing sync issues

[andrebossi]

Checklist:

• [X] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [X] I've included steps to reproduce the bug.
• [X] I've pasted the output of argocd version.

Describe the bug

I'm having errors in argocd when using the managedFieldsManagers configuration with duplicated envs in PodSpecTemplate

To Reproduce

1. Configure the argocd-cm to add ignore for managedFieldsManagers

   resource.customizations.ignoreDifferences.all: |-
   managedFieldsManagers:
   - kube-controller-manager

2. Create the application with with the resources in repository are with duplicated variables, the first creation of deployment is ok
3. Remove duplicated variables from the repository manifests and sync with removed variable
4. Get the error Sync Application and get error Failed to compare desired state to live state: failed to perform pre-diff normalization: error building typed results: error creating typedLive: .spec.template.spec.containers[name="contaienr"].env: duplicate entries for key [name="ENV_TESTE"]

Expected behavior

I expected that argocd would be able to sync the application even with duplicate variables because kubernetes only blocks this operation if applied with Server Side Apply, but even when applied without serverSideApply, argocd returns an error.

Version: I tested it on the latest stable version and this still happens

{
    "Version": "v2.10.0+2175939",
    "BuildDate": "2024-02-06T14:28:43Z",
    "GitCommit": "2175939ed6156ddd743e60f427f7f48118c971bf",
    "GitTreeState": "clean",
    "GoVersion": "go1.21.3",
    "Compiler": "gc",
    "Platform": "linux/amd64",
    "KustomizeVersion": "v5.2.1 2023-10-19T20:13:51Z",
    "HelmVersion": "v3.14.0+g3fc9f4b",
    "KubectlVersion": "v0.26.11",
    "JsonnetVersion": "v0.20.0"
}

Additional Info: I investigated this problem in argocd and found that it does not accept this because the functions in the lines below do not have the allowDuplicates option. I performed a test and compiled argocd with this option and the duplicate variables error does not occur.

https://github.com/argoproj/argo-cd/blob/2175939ed6156ddd743e60f427f7f48118c971bf/util/argo/managedfields/managed_fields.go#L94 https://github.com/argoproj/argo-cd/blob/2175939ed6156ddd743e60f427f7f48118c971bf/util/argo/managedfields/managed_fields.go#L99

[brunokino]

This discussion is related to this Issue: https://github.com/argoproj/argo-cd/discussions/17784

[agaudreault]

You mention

I expected that argocd would be able to sync the application even with duplicate variables because kubernetes only blocks this operation if applied with Server Side Apply, but even when applied without serverSideApply, argocd returns an error.

Is there a way to make it fail the first time when argocd will sync without server-side apply?

Or does this happen when existing live manifest not deployed from Argo have duplicated entries in their spec?

[brunokino]

@agaudreault ,

The problem occurs with live manifest created before adding the mentioned configuration

resource.customizations.ignoreDifferences.all: |-
  managedFieldsManagers:
  - kube-controller-manager

This configuration is manually added by ScaleOps, which is a continuous workload optimization tool.

However, once the manifest with duplicate variable exists and the configuration has been applied, users are not able to sync their applications and even remove the duplicate variable by themselves via GitOps, it needs an admin to remove it from the live manifest first.

In our company, we had to make a script to remove duplicates from all objects via kubectl. After that, users stop getting the TypedLive error and start getting a TypeConfig error during sync, but this time they are able to remove the duplication in git and continue with the Sync.

[agaudreault]

So from my understanding, if Kubernetes does not return an error on duplicated values and it is a valid live manifest, I think Argo should handle to diff and sync manifest with duplicated values.

If you can open a pull request, I think it will be a good contribution 👍