search

st-tech / gatling-operator

Automating distributed Gatling load testing using Kubernetes operator
MIT License
70 stars 19 forks source link

Support Max Concurrent Reconciles Option for Gatling Manager Controller #51

Closed yokawasa closed 2 years ago

yokawasa commented 2 years ago

Description

Support https://github.com/st-tech/gatling-operator/issues/44

With this update, the maximum number of concurrent reconciles for Gatling controller (manager) can be configured with --max-concurrent-reconciles option.

Here is the sample of Gatling controller (manager) manifest:

...
      - args:
        - --health-probe-bind-address=:8081
        - --metrics-bind-address=127.0.0.1:8080
        - --leader-elect
        - --max-concurrent-reconciles=1  # <<<<< THIS!!
        command:
        - /manager
        image: <gatling-operator-image>
...

Note

I'll add a relevant doc in another PR

Test

testing with --max-concurrent-reconciles=1 option (default)

Configure --max-concurrent-reconciles=1 like this

vi gatling-operator/config/default/manager_auth_proxy_patch.yaml

...
        args:
        - "--health-probe-bind-address=:8081"
        - "--metrics-bind-address=127.0.0.1:8080"
        - "--leader-elect"
        - "--max-concurrent-reconciles=1"

Deploy the gatling controller to the cluster (KiND) and see the controller log

# deploy to the cluster
make kind-deploy
# check the controller log
kubectl logs gatling-operator-controller-manager-5dc6f7546c-4w254  -n gatling-system -c manager

I was able to see the "worker count": 1 as expected (= means max-concurrency-reconciles==1) 

2022-05-25T10:29:45.061Z        INFO    controller-runtime.manager.controller.gatling   Starting workers        {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "worker count": 1}

After that, apply the 3 different Gatling CRs named gatling-sample01, gatling-sample02, and gatling-sample03 to the cluster, then confirmed that gatling-sample01, gatling-sample02, and gatling-sample03 were handled by a single worker from the controller log

testing with --max-concurrent-reconciles=3 option

Configure --max-concurrent-reconciles=3 like this

vi gatling-operator/config/default/manager_auth_proxy_patch.yaml

...
        args:
        - "--health-probe-bind-address=:8081"
        - "--metrics-bind-address=127.0.0.1:8080"
        - "--leader-elect"
        - "--max-concurrent-reconciles=3"

Deploy the gatling controller to the cluster (KiND) and see the controller log

# deploy to the cluster
make kind-deploy
# check the controller log
kubectl logs gatling-operator-controller-manager-5dc6f7546c-4w254  -n gatling-system -c manager

I was able to see the "worker count": 3 as expected (= means max-concurrency-reconciles==3) 

2022-05-25T10:29:45.061Z        INFO    controller-runtime.manager.controller.gatling   Starting workers        {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "worker count": 3}

After that, apply the 3 different Gatling CRs named gatling-sample01, gatling-sample02, and gatling-sample03 to the cluster, then confirmed that gatling-sample01, gatling-sample02, and gatling-sample03 were handled by a multile worker.

Here are logs of Gatling controller that handles Gatling CRs. 

2022-05-25T10:34:47.887Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Check if the runner job has completed {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample03", "namespace": "default", "namespace": "default", "name": "gatling-sample03-runner"}2022-05-25T10:34:47.887Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Runner job is still running ( Job status: active=0 failed=0 succeeded=0 ) {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample03", "namespace": "default"}2022-05-25T10:34:48.504Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Reconciling Gatling {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample01", "namespace": "default"}2022-05-25T10:34:48.505Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Check if the runner job has completed {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample01", "namespace": "default", "namespace": "default", "name": "gatling-sample01-runner"}2022-05-25T10:34:48.505Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Runner job is still running ( Job status: active=0 failed=0 succeeded=0 ) {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample01", "namespace": "default"}2022-05-25T10:34:50.623Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Reconciling Gatling {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample02", "namespace": "default"}2022-05-25T10:34:50.624Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Check if the runner job has completed {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample02", "namespace": "default", "namespace": "default", "name": "gatling-sample02-runner"}2022-05-25T10:34:50.624Z  INFO  controller-runtime.manager.controller.gatling.gatling.Reconcile Runner job is still running ( Job status: active=0 failed=0 succeeded=0 ) {"reconciler group": "gatling-operator.tech.zozo.com", "reconciler kind": "Gatling", "name": "gatling-sample02", "namespace": "default"}

As you can see, 3 different Gatling CRs (01, 02, and 03) are handled concurrently == loops for Gatling CRs occured within 3 sec.

yokawasa commented 2 years ago

@akitok @ksudate thanks much for the review!