argoproj / argo-cd

Declarative Continuous Deployment for Kubernetes
https://argo-cd.readthedocs.io
Apache License 2.0
16.73k stars 5.07k forks source link

Application tree view becomes unresponsive for big deployments #10253

Open unlightable opened 1 year ago

unlightable commented 1 year ago

We are using ArgoCD to rule over Agones. That use case implies thousands active pods and hundreds pods per second churn.

Around 100 created GameServers (and therefore pods) "application details tree" updates (I'm guessing DOM updates) start to lag behind cluster state change event stream and web page degrades to unresponsive state.

9b7f90dc4b31c97246ed21d9723062f26150685dc 1

Filtering out pods (and owning resources) through e.g. KINDS filter "collapses" the tree and everything seems to work fine after this.

Could there be a way to configure application to never show some tree subsets? Or automatically limit amount of tree children to configurable value?

I'm quite sure optimizing performance won't be enough as scaling out to tens of thousands of pods will still break things.

Example profile of main thread stalls from screenshot above. Profile-20220809T182652.zip

crenshaw-dev commented 1 year ago

I'm quite sure optimizing performance won't be enough as scaling out to tens of thousands of pods

Agreed... at some point the UI (and API server?) have to decide "this is too large to display in the normal way."

jemag commented 1 year ago

Just going to add this here for people looking into UI unresponsiveness, but in our case we had a very modest deployment but it was still being extremely unresponsive.

In our situation, we deployed Kyverno but it was not properly configured to ignore aggregated cluster roles and to track application using annotation (kyverno-install-notes)

This was causing constant refreshes of the application by the application controller: image Which would in turn result in very unresponsive UI when entering the Kyverno application.

So for some people it might be worth investigating.

Although I totally agree that, in the case of very large deployments, the UI should adapt the way it is displaying information.