No default route on pods

[845630340]

Expected Behavior

To have default route on pods so that they can connect to other worker nodes' pods.

[root@worker-2 ~]# kubectl exec -ti test-deploy -n test -- ip r
default via 172.24.68.1 dev eth0
172.24.68.0/22 dev eth0 scope link  src 172.24.68.116 

Current Behavior

There is no default route on pods. So these pods on different worker nodes can not request with each other.

[root@worker-2 ~]# kubectl exec -ti test-deploy -n test -- ip r
172.24.68.0/22 dev eth0 scope link  src 172.24.68.116 

Possible Solution

Steps to Reproduce (for bugs)

1. I deployed Flannel v0.22.3 with systemd. Its network configurate is stored in Etcd:

   
   key : /flannel/network/config

{ "Network": "172.24.0.0/13", "SubnetLen": 22, "Backend": { "Type": "vxlan", "VNI": 1 } }

2. I also deployed cri-dockerd v0.3.4 with systemd. Its parameters is "--network-plugin=cni --cni-bin-dir=/opt/cni/bin --cni-cache-dir=/var/lib/cni/cache --cni-conf-dir=/etc/cni/net.d --pod-infra-container-image=xxx/kubernetes/pause:latest"

[root@worker-2 ~]# cat /etc/cni/net.d/01-cri-dockerd.json { "cniVersion": "0.4.0", "name": "dbnet", "type": "bridge", "bridge": "docker0", "ipam": { "type": "host-local", "subnet": "172.24.68.0/22", "gateway": "172.24.68.1" } }

## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
I tried adding the following configuration to etcd, but it didn't work.

key : /flannel/network/netconfig

{ "name": "cbr0", "type": "flannel", "cniVersion": "0.4.0", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }

## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Flannel version: 0.22.3
* Backend used (e.g. vxlan or udp): vxlan
* Etcd version: 3.5.4
* Kubernetes version (if used): 1.28.2
* Operating System and version: Red Hat Enterprise Linux release 9.2 (Plow)
* Kernel: Linux jsy-30-2 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Apr 12 10:45:03 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

[thomasferrandiz]

I am confused by your setup. Are you using kubernetes or flannel standalone?

If you use, k8s, the etcd backend is not used by flannel so anything written there will be ignored. If you deploy on k8s, the best way to deploy flannel is to use the manifest provided in the repo.

[845630340]

Hi, thanks for your response!

I have experience using kubernetes v1.18 with flannel v0.11.0 as its CNI plugin. The flannel was also deployed using the systemd. I have been storing Flannel's network configuration in etcd, and actually it has been working well. It can successfully store the subnet for each worker node in etcd and set the default route for pods.

This time, I want to upgrade my Kubernetes cluster comprehensively, so I followed my previous experience. However, maybe the version span of Flannel upgrade this time is very large, and I may have missed some important update notifications.

Perhaps I should follow your suggestion and try installing flannel using manifests.

[845630340]

I have deployed flannel using the manifest, and it works fine currently.

Note that you need to configure the pod cidr related parameters in the kube-controller-manager.