Benchmarking Tooling

[Enclavet]

Summary

Add benchmarking tooling similar to the gen-resources command. The benchmarking tool can be utilized to run sync tests similar to the ones run in the scalability sig except that the tooling would be all encompassing from building the benchmarking environment, performing the benchmark, and returning a result.

Motivation

Give users the ability to perform benchmarks on their own environments to test different scalability/performance parameters.

Proposal

The tooling will be built under the hack/ directory similar to the gen-resources command. It will be split into two different parts:

Generate Benchmarking Environment Parameters:

• Num of apps
• Distribution of apps per cluster (random or equal)

What does it do?

• Create a gitea repo in k8s
• Create applications using the gitea repo as a base on the Clusters connected to a specific Argo CD installation.

Benchmarking One test will be included at the beginning, but different tests can be added as needed.

Sync test

• Will push a git change on the gitea repo to trigger a sync on the applications created in the benchmarking environment.
• Will measure when all the applications have been synced.
• Returns a sync time.

Example Command Runs:

Sync Test

ubuntu@ip-172-31-29-192:~/argo-cd$ ./dist/argocd-benchmark benchmark
INFO[0000] Starting configmap/secret informers          
INFO[0000] Configmap/secret informer synced             
INFO[0008] Processing clusters from shard -1            
INFO[0008] Using filter function:  weighted-ring-hash   
INFO[0008] Using filter function:  weighted-ring-hash   
INFO[0016] Updating cluster shards                      
WARN[0017] Cluster with name=https://kubernetes.default.svc not found in cluster map. 
INFO[0018] Running Sync test.                           
Number of Clusters:         100
Number of Applications:     5000
Status Processors:          20
Operation Processors:       10
Sharding Algorithm:         weighted-ring-hash
App Controller Replicas:    10

Shard Distrubtion:
SHARD    CLUSTERS    APPS
1        9           500
2        12          500
3        11          501
4        9           497
5        11          499
6        13          499
7        8           505
8        8           501
9        10          499
10       9           499
INFO[0043] Pushing Commit.                              
INFO[0139] Starting Test.                               
INFO[0491] All Apps Synced.                             
INFO[0491] Elapsed Time: 352 secs.      

[Z3r0Sum]

@Enclavet How does this relate to: https://github.com/argoproj-labs/argo-cd-benchmarking/tree/main/tools/awsbenchmarking-nodejs, and is this "the" tool to be using here? Is there more documentation around this upstream?

[Enclavet]

The argoproj-labs repo is a separate tool used to help with running scalability tests on EKS. This tool is designed to be run in cluster and can be used in an automated fashion for regression testing.