search

kubernetes-sigs / sig-windows-tools

Repository for tools and artifacts related to the sig-windows charter in Kubernetes. Scripts to assist kubeadm and wincat and flannel will be hosted here.
Apache License 2.0
124 stars 123 forks source link

Pod calico-node-windows error ImagePullBackOff #310

Closed kkbruce closed 1 year ago

kkbruce commented 1 year ago

Describe the bug

Follow the document https://github.com/kubernetes-sigs/sig-windows-tools/blob/master/guides/guide-for-adding-windows-node.md#configuring-calico-hostprocess ((Step 3) and complete Windows join the cluster. When the Windows node is ready, see the error from the console.

~$ kubectl describe pods calico-node-windows-4ccfb --namespace kube-system
Name:             calico-node-windows-4ccfb
Namespace:        kube-system
Priority:         0
Service Account:  calico-node
Node:             win-ubk1no1mgrs/192.168.58.130
Start Time:       Tue, 16 May 2023 14:12:07 +0000
Labels:           app=calico
                  controller-revision-hash=764bdf9f7c
                  pod-template-generation=1
                  tier=node
Annotations:      <none>
Status:           Pending
IP:               192.168.8.151
IPs:
  IP:           192.168.8.151
Controlled By:  DaemonSet/calico-node-windows
Init Containers:
  install-cni:
    Container ID:
    Image:         sigwindowstools/calico-install:v3.25.1-hostprocess
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Args:
      $env:CONTAINER_SANDBOX_MOUNT_POINT/calico/install.ps1
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Environment:
      CNI_CONF_NAME:         10-calico.conflist
      CNI_NETWORK_CONFIG:    <set to the key 'cni_network_config' of config map 'calico-config-windows'>  Optional: false
      KUBERNETES_NODE_NAME:   (v1:spec.nodeName)
      CNI_MTU:               <set to the key 'veth_mtu' of config map 'calico-config-windows'>  Optional: false
      SLEEP:                 false
      K8S_SERVICE_CIDR:      10.96.0.0/12
    Mounts:
      /etc/kubeadm-config/ from kubeadm-config (rw)
      /host/etc/cni/net.d from cni-net-dir (rw)
      /host/opt/cni/bin from cni-bin-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-77867 (ro)
Containers:
  calico-node-startup:
    Container ID:
    Image:         sigwindowstools/calico-node:v3.25.1-hostprocess
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Args:
      $env:CONTAINER_SANDBOX_MOUNT_POINT/calico/node-service.ps1
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Environment:
      POD_NAME:                   calico-node-windows-4ccfb (v1:metadata.name)
      POD_NAMESPACE:              kube-system (v1:metadata.namespace)
      CNI_IPAM_TYPE:              calico-ipam
      CALICO_NETWORKING_BACKEND:  vxlan
      KUBECONFIG:                 C:/etc/cni/net.d/calico-kubeconfig
      VXLAN_VNI:                  4096
    Mounts:
      /etc/kube-calico-windows/ from calico-config-windows (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-77867 (ro)
  calico-node-felix:
    Container ID:
    Image:         sigwindowstools/calico-node:v3.25.1-hostprocess
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Args:
      $env:CONTAINER_SANDBOX_MOUNT_POINT/calico/felix-service.ps1
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Environment:
      POD_NAME:       calico-node-windows-4ccfb (v1:metadata.name)
      POD_NAMESPACE:  kube-system (v1:metadata.namespace)
      VXLAN_VNI:      4096
      KUBECONFIG:     C:/etc/cni/net.d/calico-kubeconfig
    Mounts:
      /etc/kube-calico-windows/ from calico-config-windows (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-77867 (ro)
Conditions:
  Type              Status
  Initialized       False
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  calico-config-windows:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      calico-config-windows
    Optional:  false
  cni-bin-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /opt/cni/bin
    HostPathType:
  cni-net-dir:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/cni/net.d
    HostPathType:
  kubeadm-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      kubeadm-config
    Optional:  false
  kube-api-access-77867:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 :NoSchedule op=Exists
                             :NoExecute op=Exists
                             CriticalAddonsOnly op=Exists
                             node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/network-unavailable:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  116m                  default-scheduler  Successfully assigned kube-system/calico-node-windows-4ccfb to win-ubk1no1mgrs
  Normal   Pulling    113m (x4 over 114m)   kubelet            Pulling image "sigwindowstools/calico-install:v3.25.1-hostprocess"
  Warning  Failed     113m (x4 over 114m)   kubelet            Error: ErrImagePull
  Warning  Failed     112m (x6 over 114m)   kubelet            Error: ImagePullBackOff
  Normal   BackOff    109m (x20 over 114m)  kubelet            Back-off pulling image "sigwindowstools/calico-install:v3.25.1-hostprocess"
  Normal   Pulling    20m (x4 over 22m)     kubelet            Pulling image "sigwindowstools/calico-install:v3.25.1-hostprocess"
  Warning  Failed     20m (x4 over 22m)     kubelet            Error: ErrImagePull
  Normal   BackOff    2m1s (x86 over 21m)   kubelet            Back-off pulling image "sigwindowstools/calico-install:v3.25.1-hostprocess"

To Reproduce

$ export CALICO_VERSION="v3.25.1"
$ curl -L https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/kube-proxy/kube-proxy.yml | sed 's/KUBE_PROXY_VERSION/v1.26.4/g' | kubectl apply -f -
$ curl -L https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/calico.yml | sed "s/CALICO_VERSION/$CALICO_VERSION/g" | kubectl apply -f -
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/kube-calico-rbac.yml

Expected behavior

calico-node-windows-xxx is ready.

Kubernetes (please complete the following information):

$ kubectl get nodes -o wide
NAME              STATUS   ROLES           AGE    VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                                    KERNEL-VERSION      CONTAINER-RUNTIME
lnode             Ready    control-plane   6d1h   v1.26.4   192.168.8.150    <none>        Ubuntu 22.04.2 LTS                          5.15.0-71-generic   containerd://1.6.21
win-ubk1no1mgrs   Ready    <none>          134m   v1.26.4   192.168.58.130   <none>        Windows Server 2022 Datacenter Evaluation   10.0.20348.1726     containerd://1.6.21
$ calicoctl version
Client Version:    v3.25.1
Git commit:        82dadbce1
Cluster Version:   v3.25.1
Cluster Type:      typha,kdd,k8s,operator,bgp,kubeadm,win

Additional context None.

fabi200123 commented 1 year ago

The images for calico v3.25.1 were not build. The last version for calico-node for Windows is v3.25.0. You can check the available images for this here.

kkbruce commented 1 year ago

@fabi200123 thanks for your reply. Let me try it.