Frequent git timeouts via Application after using ApplicationSet

[chenyanshan]

Checklist:

• [*] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [*] I've included steps to reproduce the bug.
• [*] I've pasted the output of argocd version.

Describe the bug

I've created 40 Applications using 1 ApplicationSet (Previously, the same number of applications were created with the same number of applications, without these problems.), which uses the same git repository as the Application. Before using ApplicationSet, every click on the sync triggered quickly, but after using ApplicationSet, it took me an average of 3 or 4 clicks to sync properly, with a lot of time blocking in between.

To Reproduce

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: service-template
  namespace: argocd
spec:
  generators:
   - matrix:
      generators:
      - git:
          repoURL: https://xxxx.com/cicd/argocd-helm-values.git
          revision: HEAD
          directories:
          - path: common/*
      - list:
          elements:
          - cluster: xxxxxx
            address: https://xxxxxx
            env: prod
  template:
    metadata:
      name: '{{ cluster }}.{{ path.basename }}'
    spec:
      project: project
      syncPolicy:
        automated: null
        syncOptions:
          - CreateNamespace=true
      source:
        path: helm-repo/java-ms-helm/
        repoURL: 'https://xxxx.com/cicd/argocd-helm-values.git'
        targetRevision: master
        helm:
          valueFiles:
            - ../../common/{{ path.basename }}/values.yaml
            - ../../tagUpdate/{{ env }}/{{ path.basename }}-values.yaml
            - ../../multiClusterConfigure/{{ cluster }}/{{ path.basename }}-values.yaml
      destination:
        server: '{{ address }}'
        namespace: project

Expected behavior

Every time you click Sync in ArgoCD, it triggers quickly. Now every click blocks and prompts a connection to git timeout, and it takes 5-10 minutes to sync a single Application each time.

I don't know if my usage is wrong or what, but my first thought was to turn off the ApplicationSet's own sync feature and trigger the sync in some other way.

Screenshots

This problem occurs very, very frequently when Application sync

Version

v2.6.2

Logs

Paste any relevant application logs here.

[chenyanshan]

After half an hour of modifying helm and syncing all apps, there are still so many applications stuck in git failures

[blakepettersson]

There have been a bunch of performance related fixes in the ApplicationSet since 2.8, can you try that to see if this still persists?