search

turbonomic / kubeturbo

140 stars 75 forks source link

[TRB-42588]Update kubeturbo CRD and operator to suppor dynamic logging level #904

Closed kevinwangcn closed 1 year ago

kevinwangcn commented 1 year ago

Intent

Improve Kubeturbo operator to support dynamic logging level

Background

https://jsw.ibm.com/browse/TRB-42588

Testing

Case 1: Create the CR without specifying the logging level

1. Create a Kubeturbo CR with the following yaml

apiVersion: charts.helm.k8s.io/v1
kind: Kubeturbo
metadata:
  name: kubeturbo-release
spec:
  args:
    failVolumePodMoves: "false"
    kubelethttps: true
    kubeletport: 10999
  image:
    pullPolicy: Always
    repository: kevin0204/kubeturbo
    tag: loadcm
  serverMeta:
    turboServer: http://9.46.115.171
    version: "8.9"
  targetConfig:
    targetName: kind-test-cluster

2. Log into the container and check if the turbo-dynamic.config file is created or not

[kevinw@Kevins-MacBook-Pro.local kubeturbo-operator]$ k exec -it kubeturbo-release-5f85bf6fbd-5gkf6 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.4$ cat /etc/kubeturbo/turbo-dynamic.config
{
  "logging": {
    "level": 2
  }
}

Case 2: Adding the logging level configuration in the CR

1. Edit the Kubeturbo CR and add the logging level as 4 image 2. Check if the new logging level is applied and if the pod is restarted

[kevinw@Kevins-MacBook-Pro.local kubeturbo-operator]$ k logs -f kubeturbo-release-5f85bf6fbd-5gkf6 |grep -i logging
I0711 16:00:42.827855       1 kubeturbo_builder.go:1045] Logging level is changed from 2 to 4

Case 3: Openshift Operatorhub

1.Create the Kubeturbo operator from Operaterhub, and set the logging level to 5 image 2.Check the configmap, expect to see the logging level is set to 5 in the turbo-autoreload section

[root@api.ocp411kev.cp.fyre.ibm.com ~]# k get cm turbo-config-kubeturbo-release -o yaml
apiVersion: v1
data:
  turbo-autoreload.config: |-
    {
      "logging": {
        "level": 5  <-------------here
      }
    }
  turbo.config: |-
    {
      "communicationConfig": {
        "serverMeta": {
          "version": "8.9.5",
          "turboServer": "https://9.46.115.171"
        },
        "restAPIConfig": {
          "opsManagerUserName": "administrator",
          "opsManagerPassword": "administrator"
        },
...

3.check the log of the Kubeturbo pod, grep the output with the keyword Adding, those messages has the logging verbosity as 5 in the code

[root@api.ocp411kev.cp.fyre.ibm.com ~]# k logs -f kubeturbo-release-6d7fb94747-6r9gb |grep Adding
I0718 01:23:57.982973       1 node_entity_dto_builder.go:253] Adding label commodity for Node master2.ocp411kev.cp.fyre.ibm.com with key : kubernetes.io/hostname=master2.ocp411kev.cp.fyre.ibm.com
I0718 01:23:57.982979       1 node_entity_dto_builder.go:253] Adding label commodity for Node master2.ocp411kev.cp.fyre.ibm.com with key : kubernetes.io/os=linux
I0718 01:23:57.982985       1 node_entity_dto_builder.go:253] Adding label commodity for Node master2.ocp411kev.cp.fyre.ibm.com with key : node-role.kubernetes.io/master=
...

4. Change the logging level to 2 in the Kubeturbo CR, expect to see the logging level updated on the fly

[root@api.ocp411kev.cp.fyre.ibm.com ~]# k logs -f kubeturbo-release-6d7fb94747-6r9gb |grep -i logging
I0718 01:23:12.295480       1 kubeturbo_builder.go:1045] Logging level is changed from 2 to 5
I0718 01:28:24.660296       1 kubeturbo_builder.go:1045] Logging level is changed from 5 to 2

Checklist

These are the items that must be done by the developer and by reviewers before the change is ready to merge. Please strikeout any items that are not applicable, but don't delete them

Audience

(@ mention any review/... groups or people that should be aware of this merge request)

tian-ma commented 1 year ago

Depends on (https://github.com/turbonomic/kubeturbo/pull/900)