search

ppc64le-cloud / test-infra

Apache License 2.0
6 stars 20 forks source link

calico pods are not coming up #460

Closed chetjain closed 2 months ago

chetjain commented 5 months ago

[root@fvtlpar3 ~]# kubectl get pods NAME READY STATUS RESTARTS AGE calico-kube-controllers-658d97c59c-fwnv5 1/1 Running 0 10m calico-node-h4ft8 0/1 Running 0 5m47s calico-node-rm7bq 0/1 Running 0 10m coredns-5dd5756b68-dfzwx 1/1 Running 0 10m coredns-5dd5756b68-zmkfh 1/1 Running 0 10m etcd-fvtlpar3.isst.aus.stglabs.ibm.com 1/1 Running 0 10m kube-apiserver-fvtlpar3.isst.aus.stglabs.ibm.com 1/1 Running 0 10m kube-controller-manager-fvtlpar3.isst.aus.stglabs.ibm.com 1/1 Running 0 10m kube-proxy-5dvzh 1/1 Running 0 5m47s kube-proxy-dnrq5 1/1 Running 0 10m kube-scheduler-fvtlpar3.isst.aus.stglabs.ibm.com 1/1 Running 0 10m stress-ng-deployment-854cbb95db-zx695 1/1 Running 0 116s

Describing the pod, is giving below errors

Events: Type Reason Age From Message

Normal Scheduled 9m56s default-scheduler Successfully assigned kube-system/calico-node-rm7bq to fvtlpar3.isst.aus.stglabs.ibm.com Normal Pulling 9m55s kubelet Pulling image "docker.io/calico/cni:v3.25.0" Normal Pulled 9m52s kubelet Successfully pulled image "docker.io/calico/cni:v3.25.0" in 2.887s (2.887s including waiting) Normal Created 9m52s kubelet Created container upgrade-ipam Normal Started 9m52s kubelet Started container upgrade-ipam Normal Pulled 9m52s kubelet Container image "docker.io/calico/cni:v3.25.0" already present on machine Normal Created 9m52s kubelet Created container install-cni Normal Started 9m52s kubelet Started container install-cni Normal Pulling 9m51s kubelet Pulling image "docker.io/calico/node:v3.25.0" Normal Pulled 9m48s kubelet Successfully pulled image "docker.io/calico/node:v3.25.0" in 2.626s (2.626s including waiting) Normal Created 9m48s kubelet Created container mount-bpffs Normal Started 9m48s kubelet Started container mount-bpffs Normal Pulled 9m48s kubelet Container image "docker.io/calico/node:v3.25.0" already present on machine Normal Created 9m48s kubelet Created container calico-node Normal Started 9m48s kubelet Started container calico-node Warning Unhealthy 9m47s kubelet Readiness probe failed: calico/node is not ready: BIRD is not ready: Error querying BIRD: unable to connect to BIRDv4 socket: dial unix /var/run/bird/bird.ctl: connect: no such file or directory Warning Unhealthy 9m45s (x2 over 9m45s) kubelet Readiness probe failed: calico/node is not ready: BIRD is not ready: Error querying BIRD: unable to connect to BIRDv4 socket: dial unix /var/run/calico/bird.ctl: connect: connection refused Warning Unhealthy 9m35s kubelet Readiness probe failed: 2024-04-02 14:35:13.184 [INFO][1197] confd/health.go 180: Number of node(s) with BGP peering established = 0 calico/node is not ready: felix is not ready: readiness probe reporting 503 Warning Unhealthy 9m25s kubelet Readiness probe failed: 2024-04-02 14:35:23.180 [INFO][2129] confd/health.go 180: Number of node(s) with BGP peering established = 0 calico/node is not ready: felix is not ready: readiness probe reporting 503 Warning Unhealthy 9m15s kubelet Readiness probe failed: 2024-04-02 14:35:33.199 [INFO][3074] confd/health.go 180: Number of node(s) with BGP peering established = 0 calico/node is not ready: felix is not ready: readiness probe reporting 503 Warning Unhealthy 9m5s kubelet Readiness probe failed: 2024-04-02 14:35:43.214 [INFO][4007] confd/health.go 180: Number of node(s) with BGP peering established = 0 calico/node is not ready: felix is not ready: readiness probe reporting 503 Warning Unhealthy 8m55s kubelet Readiness probe failed: 2024-04-02 14:35:53.204 [INFO][4981] confd/health.go 180: Number of node(s) with BGP peering established = 0 calico/node is not ready: felix is not ready: readiness probe reporting 503 Warning Unhealthy 8m45s kubelet Readiness probe failed: 2024-04-02 14:36:03.223 [INFO][5944] confd/health.go 180: Number of node(s) with BGP peering established = 0 calico/node is not ready: felix is not ready: readiness probe reporting 503 Warning Unhealthy 8m35s kubelet Readiness probe failed: 2024-04-02 14:36:13.224 [INFO][6903] confd/health.go 180: Number of node(s) with BGP peering established = 0 calico/node is not ready: felix is not ready: readiness probe reporting 503 Warning Unhealthy 4m55s (x25 over 8m26s) kubelet (combined from similar events): Readiness probe failed: 2024-04-02 14:39:53.230 [INFO][28062] confd/health.go 180: Number of node(s) with BGP peering established = 1 calico/node is not ready: felix is not ready: readiness probe reporting 503

Rajalakshmi-Girish commented 5 months ago

The pods came up after using the calico version v3.27.0. Manifest from https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml Chetan mentioned the calico-node pods still have Warning message related to BIRD when described. @chetjain Can you please paste the out of /etc/sysctl.d/kubernetes.conf file.

mkumatag commented 5 months ago

@chetjain is this issue solved or still facing this issue?

mkumatag commented 2 months ago

Its a stale issue, please reopen the issue if it still exists, Thanks.