Manual sync takes forever - until refresh.

[sagimedina1]

Checklist:

• [V] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [V] I've included steps to reproduce the bug.
• [V] I've pasted the output of argocd version.

Describe the bug When running the sync command using the CLI, sometimes it takes a very long time to return any output. (10+ Minutes) BUT, if I will run a refresh on the same app while the CLI is waiting, it almost immediately returns the output.

The behavior is inconsistent, sometimes it takes seconds, and sometimes minutes.

To Reproduce

1. Disable auto sync for the app main-app.
2. sync an app using the CLI - argocd app sync main-app

3. wait until the process is completed.

   -> it can take 10+ minutes.

BUT If I will run it like this:

1. Disable auto sync for the app main-app.
2. sync an app using the CLI - argocd app sync main-app
3. wait for 3seconds - sleep 3
4. refresh the app - argocd app get main-app --refresh

-> immediately completed a one-liner - argocd app sync main-app --grpc-web & (sleep 3 && argocd app get main-app --grpc-web --refresh) & wait

Expected behavior I expect that each time I run the sync, the amount of time should be similar and quick. It's an urgent problem because it costs us a lot of computing time on our CI system.

Screenshots

Version Using ArgoCD chart from Bitnami. repository = "https://charts.bitnami.com/bitnami" chart = "argo-cd" version = "4.2.5"

➜  ~ helm ls -n argocd
NAME    NAMESPACE   REVISION    UPDATED                                 STATUS      CHART           APP VERSION
argo-cd argocd      17          2022-11-02 16:39:40.098021 +0200 IST    deployed    argo-cd-4.2.5   2.4.15

➜  ~ argocd version --grpc-web
argocd: v2.5.0+b895da4.dirty
  BuildDate: 2022-10-25T19:05:51Z
  GitCommit: b895da457791d56f01522796a8c3cd0f583d5d91
  GitTreeState: dirty
  GoVersion: go1.19.2
  Compiler: gc
  Platform: darwin/arm64
argocd-server: v99.99.99+unknown
  BuildDate: 1970-01-01T00:00:00Z
  GoVersion: go1.18.7
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: unknown 1970-01-01T00:00:00Z
  Helm Version: v3.10.1+g9f88ccb
  Jsonnet Version: v0.18.0```

Logs I think it's related to something I saw in the logs related to grpc.method:Watch grpc.service:application.ApplicationService

I think for some reason, the CLI is Watching for something to happen, and this "something" only happens after refresh. IF LOGS ARE NEEDED - I WILL SHARE

[sagimedina1]

@todaywasawesome, why you labeled it as low? It's extremely problematic and I didn't do anything unusual. I just installed argocd using the bitami helm chart - I guess more ppl have the same problem...

[todaywasawesome]

I marked it low because it eventually comes back correct and there's a workaround available.

[juanfresia]

Maybe this is related to #10367 ?