[BUG]sysacc pod return error after create foxlake cluster

[ahjing99]

➜ ~ kbcli version Kubernetes: v1.27.3-gke.100 KubeBlocks: 0.7.0-alpha.11 kbcli: 0.7.0-alpha.11

      `kbcli cluster create  willow28-foxlake             --termination-policy=WipeOut             --monitoring-interval=0 --enable-all-logs=false --cluster-definition=foxlake --cluster-version=foxlake-0.2.0 --set cpu=100m,memory=0.5Gi,replicas=3,storage=1Gi  --namespace default `

Cluster willow28-foxlake created

k get pod
NAME                                                         READY   STATUS    RESTARTS   AGE
sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-49gmh   0/1     Error     0          27s
sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-5cq5g   0/1     Error     0          87s
sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-g6lws   0/1     Error     0          105s
sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-nqcw8   0/1     Error     0          67s
willow28-foxlake-foxlake-metadb-0                            1/1     Running   0          6m6s
willow28-foxlake-foxlake-server-0                            1/1     Running   0          6m6s
willow28-foxlake-foxlake-server-1                            1/1     Running   0          4m29s
willow28-foxlake-foxlake-server-2                            1/1     Running   0          3m11s

k logs sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-49gmh
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2003 (HY000): Can't connect to MySQL server on '10.104.0.17:11288' (111)

k describe pod sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-49gmh
Name:         sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-49gmh
Namespace:    default
Priority:     0
Node:         gke-yjtest-default-pool-cf88b660-t6bh/10.128.15.228
Start Time:   Fri, 15 Sep 2023 11:31:37 +0800
Labels:       batch.kubernetes.io/controller-uid=5a6b429f-1975-4aed-a334-98eb08f86a7e
              batch.kubernetes.io/job-name=sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn
              controller-uid=5a6b429f-1975-4aed-a334-98eb08f86a7e
              job-name=sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn
Annotations:  <none>
Status:       Failed
IP:           10.104.1.16
IPs:
  IP:           10.104.1.16
Controlled By:  Job/sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn
Containers:
  sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn:
    Container ID:  containerd://d46c2ccf73dedaf56b0ad747468b18da921cf199078cf2e9efc6f6a1e8430ebe
    Image:         mysql:8.0.30
    Image ID:      docker.io/library/mysql@sha256:3c1aab708f6e57fc4dccafe36baccc7219acfb4ec450f3fb6d370ea89409e906
    Port:          <none>
    Host Port:     <none>
    Command:
      mysql
    Args:
      -u$(MYSQL_ROOT_USER)
      -p$(MYSQL_ROOT_PASSWORD)
      -h$(KB_ACCOUNT_ENDPOINT)
      -P$(SVC_PORT)
      -e
      $(KB_ACCOUNT_STATEMENT)
    State:          Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Fri, 15 Sep 2023 11:31:37 +0800
      Finished:     Fri, 15 Sep 2023 11:31:37 +0800
    Ready:          False
    Restart Count:  0
    Environment:
      KB_ACCOUNT_STATEMENT:  DROP USER IF EXISTS kbadmin; CREATE USER IF NOT EXISTS kbadmin IDENTIFIED BY '0v9y3JRX25'; GRANT ALL PRIVILEGES ON *.* TO kbadmin;
      KB_ACCOUNT_ENDPOINT:   10.104.0.17
      MYSQL_ROOT_USER:       <set to the key 'username' in secret 'willow28-foxlake-conn-credential'>  Optional: false
      MYSQL_ROOT_PASSWORD:   <set to the key 'password' in secret 'willow28-foxlake-conn-credential'>  Optional: false
      SVC_PORT:              <set to the key 'port' in secret 'willow28-foxlake-conn-credential'>      Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vq6s5 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-vq6s5:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 kb-data=true:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  48s   default-scheduler  Successfully assigned default/sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn-49gmh to gke-yjtest-default-pool-cf88b660-t6bh
  Normal  Pulled     48s   kubelet            Container image "mysql:8.0.30" already present on machine
  Normal  Created    48s   kubelet            Created container sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn
  Normal  Started    48s   kubelet            Started container sysacc-willow28-foxlake-foxlake-server-kbadmin-vk5kn

[shanshanying]

investigated the job and found the server is not ready for connection even though it is 'running'. A better solution it to add some running or readiness probe for foxlake-server component.

[github-actions[bot]]

This issue has been marked as stale because it has been open for 30 days with no activity

[ahjing99]

This is still failing on 0.7.0-beta.4