search

apecloud / kubeblocks

KubeBlocks is an open-source control plane software that runs and manages databases, message queues and other stateful applications on K8s.
https://kubeblocks.io
GNU Affero General Public License v3.0
2.06k stars 167 forks source link

[BUG]mysql cluster is always updating after upgrade ack os version #6599

Open ahjing99 opened 7 months ago

ahjing99 commented 7 months ago

➜ ~ kbcli version Kubernetes: v1.24.6-aliyun.1 KubeBlocks: 0.8.1 kbcli: 0.8.1

This happened after I upgraded ack os version from Alibaba Cloud Linux 2.1903 (20231008) to 3.2104 (20230727) 

➜  ~ k describe node | grep Taints
Taints:             kb-data=true:NoSchedule
Taints:             kb-controller=true:NoSchedule
Taints:             kb-data=true:NoSchedule

kbcli cluster create mysqltest --cluster-definition=mysql --cluster-version=mysql-5.7.42  --set cpu=2,memory=4Gi  --termination-policy=WipeOut

The upgrade task finished with EvictPodError for the 2 mysql pods, but kb node has been upgraded

image 
➜  ~ k get pdb
NAME              MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
mysqltest-mysql   N/A             N/A               0                     170m

But after the upgrade, mysql cluster is always updating for one pod is always ContainerCreating

➜  ~ kbcli cluster describe mysqltest
Name: mysqltest  Created Time: Feb 05,2024 17:27 UTC+0800
NAMESPACE   CLUSTER-DEFINITION   VERSION        STATUS     TERMINATION-POLICY
default     mysql                mysql-5.7.42   Updating   WipeOut

Endpoints:
COMPONENT   MODE        INTERNAL                                         EXTERNAL
mysql       ReadWrite   mysqltest-mysql.default.svc.cluster.local:3306   <none>

Topology:
COMPONENT   INSTANCE            ROLE      STATUS    AZ                 NODE                                           CREATED-TIME
mysql       mysqltest-mysql-0   primary   Running   cn-zhangjiakou-c   cn-zhangjiakou.192.168.36.149/192.168.36.149   Feb 05,2024 18:03 UTC+0800
mysql       mysqltest-mysql-1   <none>    Pending   cn-zhangjiakou-c   cn-zhangjiakou.192.168.36.151/192.168.36.151   Feb 05,2024 18:39 UTC+0800

Resources Allocation:
COMPONENT   DEDICATED   CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)   STORAGE-SIZE   STORAGE-CLASS
mysql       false       2 / 2                4Gi / 4Gi               data:20Gi      kb-default-sc

Images:
COMPONENT   TYPE    IMAGE
mysql       mysql   docker.io/mysql:5.7.42

Data Protection:
BACKUP-REPO   AUTO-BACKUP   BACKUP-SCHEDULE   BACKUP-METHOD   BACKUP-RETENTION

Show cluster events: kbcli cluster list-events -n default mysqltest

➜  ~ k describe pod mysqltest-mysql-1
Name:           mysqltest-mysql-1
Namespace:      default
Priority:       0
Node:           cn-zhangjiakou.192.168.36.151/192.168.36.151
Start Time:     Mon, 05 Feb 2024 18:41:43 +0800
Labels:         app.kubernetes.io/component=mysql
                app.kubernetes.io/instance=mysqltest
                app.kubernetes.io/managed-by=kubeblocks
                app.kubernetes.io/name=mysql
                app.kubernetes.io/version=
                apps.kubeblocks.io/component-name=mysql
                controller-revision-hash=mysqltest-mysql-579698b5b8
                statefulset.kubernetes.io/pod-name=mysqltest-mysql-1
Annotations:    apps.kubeblocks.io/component-replicas: 2
Status:         Pending
IP:
IPs:            <none>
Controlled By:  StatefulSet/mysqltest-mysql
Containers:
  mysql:
    Container ID:
    Image:         docker.io/mysql:5.7.42
    Image ID:
    Port:          3306/TCP
    Host Port:     0/TCP
    Command:
      bash
      -c
      mkdir -p /var/lib/mysql/{log,binlog};chown -R mysql:root /var/lib/mysql; /scripts/docker-entrypoint.sh mysqld --server-id $(( ${KB_POD_NAME##*-} + 1)) --ignore-db-dir=lost+found --plugin-load-add=rpl_semi_sync_master=semisync_master.so --plugin-load-add=rpl_semi_sync_slave=semisync_slave.so
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  4Gi
    Requests:
      cpu:     2
      memory:  4Gi
    Environment Variables from:
      mysqltest-mysql-env      ConfigMap  Optional: false
      mysqltest-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:          mysqltest-mysql-1 (v1:metadata.name)
      KB_POD_UID:            (v1:metadata.uid)
      KB_NAMESPACE:         default (v1:metadata.namespace)
      KB_SA_NAME:            (v1:spec.serviceAccountName)
      KB_NODENAME:           (v1:spec.nodeName)
      KB_HOST_IP:            (v1:status.hostIP)
      KB_POD_IP:             (v1:status.podIP)
      KB_POD_IPS:            (v1:status.podIPs)
      KB_HOSTIP:             (v1:status.hostIP)
      KB_PODIP:              (v1:status.podIP)
      KB_PODIPS:             (v1:status.podIPs)
      KB_POD_FQDN:          $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
      MYSQL_ROOT_HOST:      %
      MYSQL_ROOT_USER:      <set to the key 'username' in secret 'mysqltest-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:  <set to the key 'password' in secret 'mysqltest-conn-credential'>  Optional: false
    Mounts:
      /etc/mysql/conf.d from mysql-config (rw)
      /scripts from scripts (rw)
      /var/lib/mysql from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lfbcs (ro)
  metrics:
    Container ID:
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1
    Image ID:
    Port:          9104/TCP
    Host Port:     0/TCP
    Command:
      /scripts/agamotto.sh
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysqltest-mysql-env      ConfigMap  Optional: false
      mysqltest-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:     mysqltest-mysql-1 (v1:metadata.name)
      KB_POD_UID:       (v1:metadata.uid)
      KB_NAMESPACE:    default (v1:metadata.namespace)
      KB_SA_NAME:       (v1:spec.serviceAccountName)
      KB_NODENAME:      (v1:spec.nodeName)
      KB_HOST_IP:       (v1:status.hostIP)
      KB_POD_IP:        (v1:status.podIP)
      KB_POD_IPS:       (v1:status.podIPs)
      KB_HOSTIP:        (v1:status.hostIP)
      KB_PODIP:         (v1:status.podIP)
      KB_PODIPS:        (v1:status.podIPs)
      KB_POD_FQDN:     $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
      DB_TYPE:         MySQL
      ENDPOINT:        localhost:3306
      MYSQL_USER:      <set to the key 'username' in secret 'mysqltest-conn-credential'>  Optional: false
      MYSQL_PASSWORD:  <set to the key 'password' in secret 'mysqltest-conn-credential'>  Optional: false
    Mounts:
      /data/mysql from data (rw)
      /opt/agamotto from agamotto-configuration (rw)
      /scripts from scripts (rw)
      /var/log/kubeblocks from log-data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lfbcs (ro)
  kb-checkrole:
    Container ID:
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.1
    Image ID:
    Ports:         3501/TCP, 50001/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      lorry
      --port
      3501
      --grpcport
      50001
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:      0
      memory:   0
    Readiness:  http-get http://:3501/v1.0/checkrole delay=0s timeout=1s period=1s #success=1 #failure=2
    Startup:    tcp-socket :3501 delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      mysqltest-mysql-env      ConfigMap  Optional: false
      mysqltest-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:                   mysqltest-mysql-1 (v1:metadata.name)
      KB_POD_UID:                     (v1:metadata.uid)
      KB_NAMESPACE:                  default (v1:metadata.namespace)
      KB_SA_NAME:                     (v1:spec.serviceAccountName)
      KB_NODENAME:                    (v1:spec.nodeName)
      KB_HOST_IP:                     (v1:status.hostIP)
      KB_POD_IP:                      (v1:status.podIP)
      KB_POD_IPS:                     (v1:status.podIPs)
      KB_HOSTIP:                      (v1:status.hostIP)
      KB_PODIP:                       (v1:status.podIP)
      KB_PODIPS:                      (v1:status.podIPs)
      KB_POD_FQDN:                   $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
      KB_SERVICE_PORT:               3306
      KB_DATA_PATH:                  /var/lib/mysql
      KB_BUILTIN_HANDLER:            mysql
      KB_SERVICE_USER:               <set to the key 'username' in secret 'mysqltest-conn-credential'>  Optional: false
      KB_SERVICE_PASSWORD:           <set to the key 'password' in secret 'mysqltest-conn-credential'>  Optional: false
      KB_RSM_ACTION_SVC_LIST:        null
      KB_RSM_ROLE_UPDATE_MECHANISM:  DirectAPIServerEventUpdate
      KB_RSM_ROLE_PROBE_TIMEOUT:     1
      KB_CLUSTER_NAME:                (v1:metadata.labels['app.kubernetes.io/instance'])
      KB_COMP_NAME:                   (v1:metadata.labels['apps.kubeblocks.io/component-name'])
      KB_SERVICE_CHARACTER_TYPE:     mysql
    Mounts:
      /var/lib/mysql from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lfbcs (ro)
  config-manager:
    Container ID:
    Image:         infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.1
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      env
    Args:
      PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$(TOOLS_PATH)
      /bin/reloader
      --log-level
      info
      --operator-update-enable
      --tcp
      9901
      --config
      /opt/config-manager/config-manager.yaml
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      mysqltest-mysql-env      ConfigMap  Optional: false
      mysqltest-mysql-rsm-env  ConfigMap  Optional: false
    Environment:
      KB_POD_NAME:            mysqltest-mysql-1 (v1:metadata.name)
      KB_POD_UID:              (v1:metadata.uid)
      KB_NAMESPACE:           default (v1:metadata.namespace)
      KB_SA_NAME:              (v1:spec.serviceAccountName)
      KB_NODENAME:             (v1:spec.nodeName)
      KB_HOST_IP:              (v1:status.hostIP)
      KB_POD_IP:               (v1:status.podIP)
      KB_POD_IPS:              (v1:status.podIPs)
      KB_HOSTIP:               (v1:status.hostIP)
      KB_PODIP:                (v1:status.podIP)
      KB_PODIPS:               (v1:status.podIPs)
      KB_POD_FQDN:            $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
      CONFIG_MANAGER_POD_IP:   (v1:status.podIP)
      DB_TYPE:                mysql
      MYSQL_USER:             <set to the key 'username' in secret 'mysqltest-conn-credential'>  Optional: false
      MYSQL_PASSWORD:         <set to the key 'password' in secret 'mysqltest-conn-credential'>  Optional: false
      DATA_SOURCE_NAME:       $(MYSQL_USER):$(MYSQL_PASSWORD)@(localhost:3306)/
      TOOLS_PATH:             /opt/kb-tools/reload/mysql-replication-config:/opt/config-manager
    Mounts:
      /etc/mysql/conf.d from mysql-config (rw)
      /opt/config-manager from config-manager-config (rw)
      /opt/kb-tools/reload/mysql-replication-config from cm-script-mysql-replication-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lfbcs (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  data-mysqltest-mysql-1
    ReadOnly:   false
  agamotto-configuration:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      mysqltest-mysql-agamotto-configuration
    Optional:  false
  mysql-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      mysqltest-mysql-mysql-replication-config
    Optional:  false
  scripts:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      mysqltest-mysql-mysql-scripts
    Optional:  false
  cm-script-mysql-replication-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      sidecar-oracle-mysql8.0-reload-script-mysqltest
    Optional:  false
  config-manager-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      sidecar-mysqltest-mysql-config-manager-config
    Optional:  false
  log-data:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  kube-api-access-lfbcs:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 kb-data=true:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason       Age                 From     Message
  ----     ------       ----                ----     -------
  Warning  FailedMount  53m (x6 over 84m)   kubelet  Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[mysql-config scripts kube-api-access-lfbcs agamotto-configuration log-data cm-script-mysql-replication-config config-manager-config data]: timed out waiting for the condition
  Warning  FailedMount  32m (x5 over 89m)   kubelet  Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[data mysql-config scripts kube-api-access-lfbcs agamotto-configuration log-data cm-script-mysql-replication-config config-manager-config]: timed out waiting for the condition
  Warning  FailedMount  12m (x5 over 66m)   kubelet  Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[agamotto-configuration log-data cm-script-mysql-replication-config config-manager-config data mysql-config scripts kube-api-access-lfbcs]: timed out waiting for the condition
  Warning  FailedMount  8m (x50 over 93m)   kubelet  MountVolume.WaitForAttach failed for volume "d-8vbjbt4dcatjw0ip37g5" : volume attachment is being deleted
  Warning  FailedMount  3m9s (x5 over 75m)  kubelet  Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[scripts kube-api-access-lfbcs agamotto-configuration log-data cm-script-mysql-replication-config config-manager-config data mysql-config]: timed out waiting for the condition

➜  ~ k describe rsm mysqltest-mysql
Name:         mysqltest-mysql
Namespace:    default
Labels:       app.kubernetes.io/component=mysql
              app.kubernetes.io/instance=mysqltest
              app.kubernetes.io/managed-by=kubeblocks
              app.kubernetes.io/name=mysql
              apps.kubeblocks.io/component-name=mysql
Annotations:  config.kubeblocks.io/tpl-agamotto-configuration: mysqltest-mysql-agamotto-configuration
              config.kubeblocks.io/tpl-mysql-replication-config: mysqltest-mysql-mysql-replication-config
              config.kubeblocks.io/tpl-mysql-scripts: mysqltest-mysql-mysql-scripts
              kubeblocks.io/generation: 1
              monitor.kubeblocks.io/agamotto.headless.rsm: false
              monitor.kubeblocks.io/scrape.headless.rsm: false
API Version:  workloads.kubeblocks.io/v1alpha1
Kind:         ReplicatedStateMachine
Metadata:
  Creation Timestamp:  2024-02-05T09:27:28Z
  Finalizers:
    cluster.kubeblocks.io/finalizer
    rsm.workloads.kubeblocks.io/finalizer
  Generation:  1
  Managed Fields:
    API Version:  workloads.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:config.kubeblocks.io/tpl-agamotto-configuration:
          f:config.kubeblocks.io/tpl-mysql-replication-config:
          f:config.kubeblocks.io/tpl-mysql-scripts:
          f:kubeblocks.io/generation:
          f:monitor.kubeblocks.io/agamotto.headless.rsm:
          f:monitor.kubeblocks.io/scrape.headless.rsm:
        f:finalizers:
          .:
          v:"cluster.kubeblocks.io/finalizer":
          v:"rsm.workloads.kubeblocks.io/finalizer":
        f:labels:
          .:
          f:app.kubernetes.io/component:
          f:app.kubernetes.io/instance:
          f:app.kubernetes.io/managed-by:
          f:app.kubernetes.io/name:
          f:apps.kubeblocks.io/component-name:
        f:ownerReferences:
          .:
          k:{"uid":"bf21d25d-5124-480e-a347-3ba224cac44d"}:
      f:spec:
        .:
        f:credential:
          .:
          f:password:
            .:
            f:valueFrom:
              .:
              f:secretKeyRef:
          f:username:
            .:
            f:valueFrom:
              .:
              f:secretKeyRef:
        f:memberUpdateStrategy:
        f:podManagementPolicy:
        f:replicas:
        f:roleProbe:
          .:
          f:builtinHandlerName:
          f:failureThreshold:
          f:initialDelaySeconds:
          f:periodSeconds:
          f:roleUpdateMechanism:
          f:successThreshold:
          f:timeoutSeconds:
        f:roles:
        f:rsmTransformPolicy:
        f:selector:
        f:serviceName:
        f:template:
          .:
          f:metadata:
            .:
            f:labels:
              .:
              f:app.kubernetes.io/component:
              f:app.kubernetes.io/instance:
              f:app.kubernetes.io/managed-by:
              f:app.kubernetes.io/name:
              f:app.kubernetes.io/version:
              f:apps.kubeblocks.io/component-name:
          f:spec:
            .:
            f:affinity:
              .:
              f:nodeAffinity:
                .:
                f:preferredDuringSchedulingIgnoredDuringExecution:
              f:podAntiAffinity:
            f:containers:
            f:serviceAccountName:
            f:tolerations:
            f:volumes:
        f:updateStrategy:
          .:
          f:type:
        f:volumeClaimTemplates:
    Manager:      manager
    Operation:    Update
    Time:         2024-02-05T09:27:28Z
    API Version:  workloads.kubeblocks.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:availableReplicas:
        f:collisionCount:
        f:currentGeneration:
        f:currentReplicas:
        f:currentRevision:
        f:initReplicas:
        f:membersStatus:
        f:observedGeneration:
        f:readyInitReplicas:
        f:readyReplicas:
        f:replicas:
        f:updateRevision:
        f:updatedReplicas:
    Manager:      manager
    Operation:    Update
    Subresource:  status
    Time:         2024-02-05T09:28:08Z
  Owner References:
    API Version:           apps.kubeblocks.io/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Component
    Name:                  mysqltest-mysql
    UID:                   bf21d25d-5124-480e-a347-3ba224cac44d
  Resource Version:        34390
  UID:                     3880e115-7dfd-400f-b50f-d78a41eb91fc
Spec:
  Credential:
    Password:
      Value From:
        Secret Key Ref:
          Key:   password
          Name:  mysqltest-conn-credential
    Username:
      Value From:
        Secret Key Ref:
          Key:             username
          Name:            mysqltest-conn-credential
  Member Update Strategy:  Serial
  Pod Management Policy:   Parallel
  Replicas:                2
  Role Probe:
    Builtin Handler Name:   mysql
    Failure Threshold:      2
    Initial Delay Seconds:  0
    Period Seconds:         1
    Role Update Mechanism:  DirectAPIServerEventUpdate
    Success Threshold:      1
    Timeout Seconds:        1
  Roles:
    Access Mode:         ReadWrite
    Can Vote:            true
    Is Leader:           true
    Name:                primary
    Access Mode:         Readonly
    Can Vote:            true
    Is Leader:           false
    Name:                secondary
  Rsm Transform Policy:  ToSts
  Selector:
    Match Labels:
      app.kubernetes.io/instance:         mysqltest
      app.kubernetes.io/managed-by:       kubeblocks
      app.kubernetes.io/name:             mysql
      apps.kubeblocks.io/component-name:  mysql
  Service Name:                           mysqltest-mysql-headless
  Template:
    Metadata:
      Labels:
        app.kubernetes.io/component:        mysql
        app.kubernetes.io/instance:         mysqltest
        app.kubernetes.io/managed-by:       kubeblocks
        app.kubernetes.io/name:             mysql
        app.kubernetes.io/version:
        apps.kubeblocks.io/component-name:  mysql
    Spec:
      Affinity:
        Node Affinity:
          Preferred During Scheduling Ignored During Execution:
            Preference:
              Match Expressions:
                Key:       kb-data
                Operator:  In
                Values:
                  true
            Weight:  100
        Pod Anti Affinity:
      Containers:
        Command:
          bash
          -c
          mkdir -p /var/lib/mysql/{log,binlog};chown -R mysql:root /var/lib/mysql; /scripts/docker-entrypoint.sh mysqld --server-id $(( ${KB_POD_NAME##*-} + 1)) --ignore-db-dir=lost+found --plugin-load-add=rpl_semi_sync_master=semisync_master.so --plugin-load-add=rpl_semi_sync_slave=semisync_slave.so
        Env:
          Name:  KB_POD_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.name
          Name:             KB_POD_UID
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.uid
          Name:             KB_NAMESPACE
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.namespace
          Name:             KB_SA_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.serviceAccountName
          Name:             KB_NODENAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.nodeName
          Name:             KB_HOST_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_POD_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_POD_IPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_HOSTIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_PODIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_PODIPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_POD_FQDN
          Value:            $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
          Name:             MYSQL_ROOT_HOST
          Value:            %
          Name:             MYSQL_ROOT_USER
          Value From:
            Secret Key Ref:
              Key:       username
              Name:      mysqltest-conn-credential
              Optional:  false
          Name:          MYSQL_ROOT_PASSWORD
          Value From:
            Secret Key Ref:
              Key:       password
              Name:      mysqltest-conn-credential
              Optional:  false
        Env From:
          Config Map Ref:
            Name:           mysqltest-mysql-env
            Optional:       false
        Image:              docker.io/mysql:5.7.42
        Image Pull Policy:  IfNotPresent
        Name:               mysql
        Ports:
          Container Port:  3306
          Name:            mysql
          Protocol:        TCP
        Resources:
          Limits:
            Cpu:     2
            Memory:  4Gi
          Requests:
            Cpu:     2
            Memory:  4Gi
        Volume Mounts:
          Mount Path:  /var/lib/mysql
          Name:        data
          Mount Path:  /etc/mysql/conf.d
          Name:        mysql-config
          Mount Path:  /scripts
          Name:        scripts
        Command:
          /scripts/agamotto.sh
        Env:
          Name:  KB_POD_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.name
          Name:             KB_POD_UID
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.uid
          Name:             KB_NAMESPACE
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.namespace
          Name:             KB_SA_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.serviceAccountName
          Name:             KB_NODENAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.nodeName
          Name:             KB_HOST_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_POD_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_POD_IPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_HOSTIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_PODIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_PODIPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_POD_FQDN
          Value:            $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
          Name:             DB_TYPE
          Value:            MySQL
          Name:             ENDPOINT
          Value:            localhost:3306
          Name:             MYSQL_USER
          Value From:
            Secret Key Ref:
              Key:       username
              Name:      mysqltest-conn-credential
              Optional:  false
          Name:          MYSQL_PASSWORD
          Value From:
            Secret Key Ref:
              Key:       password
              Name:      mysqltest-conn-credential
              Optional:  false
        Env From:
          Config Map Ref:
            Name:           mysqltest-mysql-env
            Optional:       false
        Image:              infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/agamotto:0.1.2-beta.1
        Image Pull Policy:  IfNotPresent
        Name:               metrics
        Ports:
          Container Port:  9104
          Name:            http-metrics
          Protocol:        TCP
        Resources:
          Limits:
            Cpu:     0
            Memory:  0
        Security Context:
          Run As Non Root:  false
          Run As User:      0
        Volume Mounts:
          Mount Path:  /opt/agamotto
          Name:        agamotto-configuration
          Mount Path:  /data/mysql
          Name:        data
          Mount Path:  /var/log/kubeblocks
          Name:        log-data
          Mount Path:  /scripts
          Name:        scripts
        Command:
          lorry
          --port
          3501
          --grpcport
          50001
        Env:
          Name:  KB_POD_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.name
          Name:             KB_POD_UID
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.uid
          Name:             KB_NAMESPACE
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.namespace
          Name:             KB_SA_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.serviceAccountName
          Name:             KB_NODENAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.nodeName
          Name:             KB_HOST_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_POD_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_POD_IPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_HOSTIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_PODIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_PODIPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_POD_FQDN
          Value:            $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
          Name:             KB_SERVICE_PORT
          Value:            3306
          Name:             KB_DATA_PATH
          Value:            /var/lib/mysql
          Name:             KB_BUILTIN_HANDLER
          Value:            mysql
          Name:             KB_SERVICE_USER
          Value From:
            Secret Key Ref:
              Key:   username
              Name:  mysqltest-conn-credential
          Name:      KB_SERVICE_PASSWORD
          Value From:
            Secret Key Ref:
              Key:   password
              Name:  mysqltest-conn-credential
        Env From:
          Config Map Ref:
            Name:           mysqltest-mysql-env
            Optional:       false
        Image:              infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.1
        Image Pull Policy:  IfNotPresent
        Name:               kb-checkrole
        Ports:
          Container Port:  3501
          Name:            lorry-http-port
          Protocol:        TCP
          Container Port:  50001
          Name:            lorry-grpc-port
          Protocol:        TCP
        Readiness Probe:
          Failure Threshold:  2
          Http Get:
            Path:           /v1.0/checkrole
            Port:           3501
          Period Seconds:   1
          Timeout Seconds:  1
        Resources:
          Limits:
            Cpu:     0
            Memory:  0
        Startup Probe:
          Tcp Socket:
            Port:  3501
        Volume Mounts:
          Mount Path:  /var/lib/mysql
          Name:        data
        Args:
          PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$(TOOLS_PATH)
          /bin/reloader
          --log-level
          info
          --operator-update-enable
          --tcp
          9901
          --config
          /opt/config-manager/config-manager.yaml
        Command:
          env
        Env:
          Name:  KB_POD_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.name
          Name:             KB_POD_UID
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.uid
          Name:             KB_NAMESPACE
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   metadata.namespace
          Name:             KB_SA_NAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.serviceAccountName
          Name:             KB_NODENAME
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   spec.nodeName
          Name:             KB_HOST_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_POD_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_POD_IPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_HOSTIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.hostIP
          Name:             KB_PODIP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             KB_PODIPS
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIPs
          Name:             KB_POD_FQDN
          Value:            $(KB_POD_NAME).mysqltest-mysql-headless.$(KB_NAMESPACE).svc
          Name:             CONFIG_MANAGER_POD_IP
          Value From:
            Field Ref:
              API Version:  v1
              Field Path:   status.podIP
          Name:             DB_TYPE
          Value:            mysql
          Name:             MYSQL_USER
          Value From:
            Secret Key Ref:
              Key:   username
              Name:  mysqltest-conn-credential
          Name:      MYSQL_PASSWORD
          Value From:
            Secret Key Ref:
              Key:   password
              Name:  mysqltest-conn-credential
          Name:      DATA_SOURCE_NAME
          Value:     $(MYSQL_USER):$(MYSQL_PASSWORD)@(localhost:3306)/
          Name:      TOOLS_PATH
          Value:     /opt/kb-tools/reload/mysql-replication-config:/opt/config-manager
        Env From:
          Config Map Ref:
            Name:           mysqltest-mysql-env
            Optional:       false
        Image:              infracreate-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.8.1
        Image Pull Policy:  IfNotPresent
        Name:               config-manager
        Resources:
          Limits:
            Cpu:     0
            Memory:  0
        Volume Mounts:
          Mount Path:        /etc/mysql/conf.d
          Name:              mysql-config
          Mount Path:        /opt/kb-tools/reload/mysql-replication-config
          Name:              cm-script-mysql-replication-config
          Mount Path:        /opt/config-manager
          Name:              config-manager-config
      Service Account Name:  kb-mysqltest
      Tolerations:
        Effect:    NoSchedule
        Key:       kb-data
        Operator:  Equal
        Value:     true
      Volumes:
        Config Map:
          Default Mode:  292
          Name:          mysqltest-mysql-agamotto-configuration
        Name:            agamotto-configuration
        Config Map:
          Name:  mysqltest-mysql-mysql-replication-config
        Name:    mysql-config
        Config Map:
          Default Mode:  365
          Name:          mysqltest-mysql-mysql-scripts
        Name:            scripts
        Config Map:
          Default Mode:  493
          Name:          sidecar-oracle-mysql8.0-reload-script-mysqltest
        Name:            cm-script-mysql-replication-config
        Config Map:
          Default Mode:  493
          Name:          sidecar-mysqltest-mysql-config-manager-config
        Name:            config-manager-config
        Empty Dir:
        Name:  data
        Empty Dir:
        Name:  log-data
  Update Strategy:
    Type:  OnDelete
  Volume Claim Templates:
    Metadata:
      Labels:
        apps.kubeblocks.io/vct-name:  data
        kubeblocks.io/volume-type:    data
      Name:                           data
    Spec:
      Access Modes:
        ReadWriteOnce
      Resources:
        Requests:
          Storage:         20Gi
      Storage Class Name:  kb-default-sc
    Status:
Status:
  Available Replicas:  1
  Collision Count:     0
  Current Generation:  1
  Current Replicas:    2
  Current Revision:    mysqltest-mysql-579698b5b8
  Init Replicas:       2
  Members Status:
    Pod Name:               mysqltest-mysql-0
    Ready Without Primary:  false
    Role:
      Access Mode:      ReadWrite
      Can Vote:         true
      Is Leader:        true
      Name:             primary
  Observed Generation:  1
  Ready Init Replicas:  2
  Ready Replicas:       1
  Replicas:             2
  Update Revision:      mysqltest-mysql-579698b5b8
  Updated Replicas:     2
Events:                 <none>
➜  ~

➜ ~ kbcli report cluster mysqltest --with-logs --all-containers reporting cluster information to report-cluster-mysqltest-2024-02-05-20-16-42.zip processing manifests OK processing events OK process pod logs OK ➜ ~ kbcli report kubeblocks --with-logs --all-containers --output yaml reporting KubeBlocks information to report-kubeblocks-2024-02-05-20-16-53.zip processing manifests OK processing events OK process pod logs OK report-kubeblocks-2024-02-05-20-16-53.zip report-cluster-mysqltest-2024-02-05-20-16-42.zip

github-actions[bot] commented 6 months ago

This issue has been marked as stale because it has been open for 30 days with no activity