fujitatomoya / ros_k8s

Kuberenetes / ROS&ROS2 Cluster Samples
Creative Commons Attribution 4.0 International
192 stars 29 forks source link

Ubuntu 22.04 support amd64/aarch64 #21

Closed fujitatomoya closed 1 year ago

fujitatomoya commented 1 year ago

Upgrading aarch64 platform to ubuntu 22.04 led to the following unstable problem, which cannot start CNI (flannel/weave) deployment as expected.

root@tomoyafujita-HP-Compaq-Elite-8300-SFF:/home/tomoyafujita/DVT/github.com/fujitatomoya/ros_k8s/yaml# kubectl logs -n kube-flannel kube-flannel-ds-8zr2k
Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)
I0414 06:34:54.014798       1 main.go:211] CLI flags config: {etcdEndpoints:http://127.0.0.1:4001,http://127.0.0.1:2379 etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:flannel.alpha.coreos.com kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true useMultiClusterCidr:false}
W0414 06:34:54.015087       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0414 06:34:54.046028       1 kube.go:144] Waiting 10m0s for node controller to sync
I0414 06:34:54.046165       1 kube.go:485] Starting kube subnet manager
I0414 06:34:54.052649       1 kube.go:506] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.244.0.0/24]
I0414 06:34:55.047254       1 kube.go:151] Node controller sync successful
I0414 06:34:55.047812       1 main.go:231] Created subnet manager: Kubernetes Subnet Manager - ubuntu
I0414 06:34:55.048088       1 main.go:234] Installing signal handlers
I0414 06:34:55.048991       1 main.go:542] Found network config - Backend type: vxlan
I0414 06:34:55.049164       1 match.go:206] Determining IP address of default interface
I0414 06:34:55.051469       1 match.go:259] Using interface with name eth0 and address 192.168.1.238
I0414 06:34:55.051589       1 match.go:281] Defaulting external address to interface address (192.168.1.238)
I0414 06:34:55.051872       1 vxlan.go:140] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
E0414 06:34:55.061475       1 main.go:334] Error registering network: operation not supported
I0414 06:34:55.061615       1 main.go:522] Stopping shutdownHandler...

could be related https://stackoverflow.com/questions/52098214/kube-flannel-in-crashloopbackoff-status

fujitatomoya commented 1 year ago

This is duplicated issue, see the following related issues.

the reason is vxlan kernel module will not be loaded by default Ubuntu 21.10 or later. this leads CNI operation to fail. how to fix this issue is simple, just load the vxlan module to raspi4 kernel.

sudo apt install -y linux-modules-extra-raspi

I confirmed that this solves my problem here.

fujitatomoya commented 1 year ago

addressed by https://github.com/fujitatomoya/ros_k8s/commit/001cf1aec3bf36c83d3d5194ec4f0ade352c2231