--helm-set not working when using multi source applicationsets or applications

Checklist:

Describe the bug

I'm trying to install an ApplicationSet using a custom helm repo, using value files that are located in a different repo, and deploy a new version using argocd cli with the argocd app set command, overriding parameters. the issue is that it seems argocd sees the change and shows the app is out of sync, but then it syncs it automatically and it looks like it reverts back to the original values in the files, even though I have disabled the automatic sync setting for the ApplicationSet helm repo is defined correctly, and everything seems to be ok, just the app set command not bumping the version as youd expect. this is the appset yaml:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: test-python-service
  labels:
    service: test-python-service
spec:
  goTemplate: true
  goTemplateOptions:
    - missingkey=error
  generators:
    - list:
        elements:
          - cluster: staging
            account: 'ACCOUNT'
            region: REGION
  template:
    metadata:
      name: test-python-service-{{ .cluster }}
      labels:
        env: '{{ .cluster }}'
        service: test-python-service
    spec:
      project: '{{.cluster}}-core'

      sources:
        - repoURL: '{{ .account }}.dkr.ecr.{{ .region }}.amazonaws.com'
          chart: helm-charts/app
          path: helm-charts/app
          targetRevision: 1.0.0
          helm:
            releaseName: test-python-service
            valueFiles:
              - $values/base.yaml
              - $values/env/{{ .cluster }}.yaml

        - repoURL: https://github.com/org/test-python-service
          targetRevision: main
          ref: values

      destination:
        name: '{{.cluster}}'
        namespace: test-python-service
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
          allowEmpty: false
        syncOptions:
          - ApplyOutOfSyncOnly=true
  strategy:
    type: RollingSync
    rollingSync:
      steps:
        - matchExpressions:
            - key: env
              operator: In
              values:
                - staging

To Reproduce

Create an ApplicationSet with multiple sources
First Item in the sources array is the helm chart
Second Item in the sources array is the values file reference
Run a CLI command to override the contents like

argocd app set test-python-service-staging \
  --grpc-web --server $ARGOCD_SERVER  \
  --source-position 1 \
  --helm-set service.image.tag=v0.6.0

ArgoCD will report a SYNC status when the command is executed but will not override with the parameters from the CLI but instead will override with the settings from the second ref values in the sources array

Expected behavior

The final rendered template should reflect the helm values override being done by the CLI command, in this example it should set the image tag to v0.6.0 in the helm template.

Screenshots

No screenshot necessary as it returns 200 OK, argocd simply refuses to update the helm value through CLI Version

❯ argocd version --grpc-web
argocd: v2.12.0+ec30a48
  BuildDate: 2024-08-05T13:46:43Z
  GitCommit: ec30a48bce7a60046836e481cd2160e28c59231d
  GitTreeState: clean
  GoVersion: go1.22.5
  Compiler: gc
  Platform: linux/amd64
argocd-server: v2.12.1+26b2039
  BuildDate: 2024-08-16T16:42:13Z
  GitCommit: 26b2039a55b9bdf807a70d344af8ade5171d3d39
  GitTreeState: clean
  GoVersion: go1.22.4
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: v5.4.2 2024-05-22T15:19:38Z
  Helm Version: v3.15.2+g1a500d5
  Kubectl Version: v0.29.6
  Jsonnet Version: v0.20.0

Logs

time="2024-08-19T15:16:46Z" level=info msg="Refreshing app status (at least one of the spec.sources differs), level (3)" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:46Z" level=info msg="Comparing app state (cluster: https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com, namespace: test-python-service)" application=argocd/test-python-service-staging
time="2024-08-19T15:16:46Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:46Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:47Z" level=info msg="GetRepoObjs stats" application=argocd/test-python-service-staging build_options_ms=0 helm_ms=434 plugins_ms=0 repo_ms=0 time_ms=625 unmarshal_ms=189 version_ms=0
time="2024-08-19T15:16:47Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" app-namespace=argocd app-qualified-name=argocd/open-policy-agent-production-ap1 application=open-policy-agent-production-ap1 project=production-ap1-core
time="2024-08-19T15:16:47Z" level=info msg="Comparing app state (cluster: https://EC11DB68ADE11D085B80CA9145EB1000.yl4.ap-southeast-2.eks.amazonaws.com, namespace: gatekeeper-system)" application=argocd/open-policy-agent-production-ap1
W0819 15:16:47.209117       7 warnings.go:70] unknown field "status.history[0].initiatedBy"
W0819 15:16:47.209148       7 warnings.go:70] unknown field "status.history[1].initiatedBy"
W0819 15:16:47.209153       7 warnings.go:70] unknown field "status.history[2].initiatedBy"
time="2024-08-19T15:16:47Z" level=info msg="Initiated automated sync to '2.6.7, c2cbf3c2bf531779f23ad7201afbfda6f05e7173'" application=test-python-service-staging dest-namespace=test-python-service dest-server="https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com" reason=OperationStarted type=Normal
time="2024-08-19T15:16:47Z" level=info msg="Initiated automated sync to '2.6.7, c2cbf3c2bf531779f23ad7201afbfda6f05e7173'" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Updated sync status: Synced -> OutOfSync" application=test-python-service-staging dest-namespace=test-python-service dest-server="https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com" reason=ResourceUpdated type=Normal
time="2024-08-19T15:16:47Z" level=info msg="GetRepoObjs stats" application=argocd/open-policy-agent-production-ap1 build_options_ms=0 helm_ms=2 plugins_ms=0 repo_ms=0 time_ms=68 unmarshal_ms=9 version_ms=0
time="2024-08-19T15:16:47Z" level=info msg="updated 'argocd/test-python-service-staging' operation (phase: Running)" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Initialized new operation: {&SyncOperation{Revision:,Prune:true,DryRun:false,SyncStrategy:nil,Resources:[]SyncOperationResource{},Source:nil,Manifests:[],SyncOptions:[ApplyOutOfSyncOnly=true],Sources:[]ApplicationSource{},Revisions:[2.6.7 c2cbf3c2bf531779f23ad7201afbfda6f05e7173],} { true} [] {5 nil}}" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Comparing app state (cluster: https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com, namespace: test-python-service)" application=argocd/test-python-service-staging
time="2024-08-19T15:16:47Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:47Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:47Z" level=info msg="Update successful" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Reconciliation completed" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging comparison-level=3 dedup_ms=0 dest-name=staging dest-namespace=test-python-service dest-server="https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com" diff_ms=65 git_ms=625 health_ms=1 live_ms=7 patch_ms=36 project=staging-core setop_ms=44 settings_ms=0 sync_ms=0 time_ms=864
time="2024-08-19T15:16:47Z" level=info msg="Refreshing app status (at least one of the spec.sources differs), level (3)" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Comparing app state (cluster: https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com, namespace: test-python-service)" application=argocd/test-python-service-staging
time="2024-08-19T15:16:47Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:47Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:47Z" level=info msg="GetRepoObjs stats" application=argocd/test-python-service-staging build_options_ms=0 helm_ms=303 plugins_ms=0 repo_ms=0 time_ms=308 unmarshal_ms=1 version_ms=0
time="2024-08-19T15:16:47Z" level=info msg="GetRepoObjs stats" application=argocd/test-python-service-staging build_options_ms=0 helm_ms=185 plugins_ms=0 repo_ms=0 time_ms=328 unmarshal_ms=141 version_ms=0
time="2024-08-19T15:16:47Z" level=info msg=Syncing application=argocd/test-python-service-staging skipHooks=false started=false syncId=00012-cmwSp
time="2024-08-19T15:16:47Z" level=info msg="Tasks (dry-run)" application=argocd/test-python-service-staging syncId=00012-cmwSp tasks="[]"
time="2024-08-19T15:16:47Z" level=info msg="Updating operation state. phase: Running -> Succeeded, message: '' -> 'successfully synced (no more tasks)'" application=argocd/test-python-service-staging syncId=00012-cmwSp
time="2024-08-19T15:16:47Z" level=info msg="sync/terminate complete" application=argocd/test-python-service-staging duration=48.010389ms syncId=00012-cmwSp
time="2024-08-19T15:16:47Z" level=info msg="Skipping auto-sync: another operation is in progress" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Updated sync status: OutOfSync -> Synced" application=test-python-service-staging dest-namespace=test-python-service dest-server="https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com" reason=ResourceUpdated type=Normal
W0819 15:16:47.723657       7 warnings.go:70] unknown field "status.history[0].initiatedBy"
W0819 15:16:47.723689       7 warnings.go:70] unknown field "status.history[1].initiatedBy"
W0819 15:16:47.723696       7 warnings.go:70] unknown field "status.history[2].initiatedBy"
W0819 15:16:47.723701       7 warnings.go:70] unknown field "status.history[3].initiatedBy"
time="2024-08-19T15:16:47Z" level=info msg="updated 'argocd/test-python-service-staging' operation (phase: Succeeded)" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Sync operation to  succeeded" application=test-python-service-staging dest-namespace=test-python-service dest-server="https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com" reason=OperationCompleted type=Normal
time="2024-08-19T15:16:47Z" level=info msg="Update successful" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Reconciliation completed" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging comparison-level=3 dedup_ms=0 dest-name=staging dest-namespace=test-python-service dest-server="https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com" diff_ms=44 git_ms=329 health_ms=1 live_ms=3 patch_ms=60 project=staging-core setop_ms=0 settings_ms=0 sync_ms=0 time_ms=489
time="2024-08-19T15:16:47Z" level=info msg="Refreshing app status (controller refresh requested), level (3)" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:47Z" level=info msg="Comparing app state (cluster: https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com, namespace: test-python-service)" application=argocd/test-python-service-staging
time="2024-08-19T15:16:47Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:47Z" level=warning msg="Found multiple credentials for repoURL: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com"
time="2024-08-19T15:16:48Z" level=info msg="GetRepoObjs stats" application=argocd/test-python-service-staging build_options_ms=0 helm_ms=143 plugins_ms=0 repo_ms=0 time_ms=300 unmarshal_ms=151 version_ms=0
time="2024-08-19T15:16:48Z" level=info msg="Skipping auto-sync: application status is Synced" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:48Z" level=info msg="No status changes. Skipping patch" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging project=staging-core
time="2024-08-19T15:16:48Z" level=info msg="Reconciliation completed" app-namespace=argocd app-qualified-name=argocd/test-python-service-staging application=test-python-service-staging comparison-level=3 dedup_ms=0 dest-name=staging dest-namespace=test-python-service dest-server="https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com" diff_ms=48 git_ms=300 health_ms=1 live_ms=2 patch_ms=0 project=staging-core setop_ms=0 settings_ms=0 sync_ms=0 time_ms=370
time="2024-08-19T15:16:48Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" app-namespace=argocd app-qualified-name=argocd/datadog-operator-staging application=datadog-operator-staging project=staging-core
time="2024-08-19T15:16:48Z" level=info msg="Comparing app state (cluster: https://xxxxxxxxxxxxxxxxxx.sk1.eu-west-1.eks.amazonaws.com, namespace: monitor)" application=argocd/datadog-operator-staging

Paste any relevant application logs here.

argoproj / argo-cd

--helm-set not working when using multi source applicationsets or applications #19590