cilium ipam replaced to kube-ipam failed

[wenhuwang]

General Information

• Orchestration system version

  # kubectl version
  Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:10:45Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:04:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

• cilium version

  # cilium version
  Client: 1.8.3 54cf3810d 2020-09-04T14:01:53+02:00 go version go1.14.7 linux/amd64
  Daemon: 1.8.3 54cf3810d 2020-09-04T14:01:53+02:00 go version go1.14.7 linux/amd64

• Kernel version

  # uname -a
  Linux node1 5.11.0-41-generic #45~20.04.1-Ubuntu SMP Wed Nov 10 10:20:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

kube-ipam config

# ls -l /opt/cni/bin/
total 131796
-rwxr-xr-x 1 root root  4151672 2月   5  2021 bandwidth
-rwxr-xr-x 1 root root  4536104 2月   5  2021 bridge
-rwxr-xr-x 1 root root 13979648 12月  6 16:34 cilium-cni
-rwxr-xr-x 1 root root 10270090 2月   5  2021 dhcp
-rwxr-xr-x 1 root root  4767801 2月   5  2021 firewall
-rwxr-xr-x 1 root root  3357992 2月   5  2021 flannel
-rwxr-xr-x 1 root root  4144106 2月   5  2021 host-device
-rwxr-xr-x 1 root root  3565330 2月   5  2021 host-local
-rwxr-xr-x 1 root root  4288339 2月   5  2021 ipvlan
-rwxr-xr-x 1 root root 46967980 12月  6 16:50 kube-ipam
-rwxr-xr-x 1 root root  3530531 2月   5  2021 loopback
-rwxr-xr-x 1 root root  4367216 2月   5  2021 macvlan
-rwxr-xr-x 1 root root  3966455 2月   5  2021 portmap
-rwxr-xr-x 1 root root  4467317 2月   5  2021 ptp
-rwxr-xr-x 1 root root  3701138 2月   5  2021 sbr
-rwxr-xr-x 1 root root  3153330 2月   5  2021 static
-rwxr-xr-x 1 root root  3668289 2月   5  2021 tuning
-rwxr-xr-x 1 root root  4287972 2月   5  2021 vlan
-rwxr-xr-x 1 root root  3759977 2月   5  2021 vrf

# ll /etc/cni/net.d/
total 16
drwxr-xr-x 2 kube root 4096 12月  6 17:59 ./
drwxr-xr-x 3 kube root 4096 12月  6 16:31 ../
-rw-r--r-- 1 root root  827 12月  6 17:59 05-cilium.conf
-rw-r--r-- 1 root root   97 12月  6 16:44 05-cilium.conf.bak
# cat /etc/cni/net.d/05-cilium.conf
{
  "cniVersion": "0.3.1",
  "name": "cilium",
  "type": "cilium-cni",
  "enable-debug": true,
  "ipam": {
           "name": "kube-subnet",
           "type": "kube-ipam",
           "etcdConfig": {
                   "etcdURL": "https://127.0.0.1:2379",
                    "kubeConfig": "/etc/kubernetes/admin.conf",
                   "etcdCertFile": "/etc/ssl/etcd/ssl/admin-node1.pem",
                   "etcdKeyFile": "/etc/ssl/etcd/ssl/admin-node1-key.pem",
                   "etcdTrustedCAFileFile": "/etc/ssl/etcd/ssl/ca.pem"
           },
           "subnet": "10.188.0.0/16",
           "rangeStart": "10.188.0.10",
           "rangeEnd": "10.188.0.200",
           "gateway": "10.188.0.1",
           "routes": [{
                   "dst": "0.0.0.0/0"
           }],
           "resolvConf": "/etc/resolv.conf"
        }
}

And when i created pod, the pod ip is still uses the ip of the cilium network。

# kubectl get pods web-5c57fc9cf6-wp5b6 -owide
NAME                   READY   STATUS    RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
web-5c57fc9cf6-wp5b6   1/1     Running   0          90s   10.233.64.47   node1   <none>           <none>

[cloudnativer]

cilium is not directly supported for the time being. You can consider using cilium in combination with Multus. Refer to the following link for usage: https://github.com/cloudnativer/kube-ipam/blob/main/docs/Networksecuritylayering.md

[cloudnativer]

We have recorded your requirements and we will consider supporting them in future versions.