k8snetworkplumbingwg / multus-cni

A CNI meta-plugin for multi-homed pods in Kubernetes
Apache License 2.0
2.27k stars 577 forks source link

Many hostport iptables rule with multus cni #1254

Closed zhonglin6666 closed 1 month ago

zhonglin6666 commented 3 months ago

What happend: There is an ingress-controller in the cluster that exposes port 80 and 443 port, When restarting the Docker daemon, there may be an additional host port iptables rule. Here are the situations where multiple restarts have occurred, Restarting Docker will add a host port iptables rule

-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"k8s-pod-network\" id: \"7ad6a8ce5b6ddfe491a38d9c5333251107ebf5c97513780af28bc044a294b433\"" -m multiport --dports 80,443 -j CNI-DN-a07f9dd15b9a28dd35dd9
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"k8s-pod-network\" id: \"7452a2e1fbf05eb09405405323992785a242528657f9a054d1839dc661c1d5d1\"" -m multiport --dports 80,443 -j CNI-DN-29d62401cedb198cb7b6f
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"k8s-pod-network\" id: \"44f01b76abb4796bc8c69f7529b7eeb57a15e2ea4777698474731726c0c0d6da\"" -m multiport --dports 80,443 -j CNI-DN-7e9430b3593bbd705bde0
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"k8s-pod-network\" id: \"30df0a5c1b15236b885641e5979e5b35ce3ed4b22d613fd5cba9b9c2156520b7\"" -m multiport --dports 80,443 -j CNI-DN-988a9c68e3d6ac8c1b426
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"k8s-pod-network\" id: \"7ffcf8eadba037fa0714da79aabbc2fa1a65b4c8af5b8ec301870c6ca52de2dc\"" -m multiport --dports 80,443 -j CNI-DN-0f6d603358e1f87b6adaf
-A CNI-HOSTPORT-DNAT -p tcp -m comment --comment "dnat name: \"k8s-pod-network\" id: \"7622dbc7d4861fa6a47ad120b3fc28bf9aec13b7a42f4af7af3f2719063a3d27\"" -m multiport --dports 80,443 -j CNI-DN-14bb0c231b4518a1d8a9d

Not using the multus-cni plugin, restarting Docker with only one hostport iptables rule.

What you expected to happen: There is only one host port iptables rule.

How to reproduce it (as minimally and precisely as possible):

  1. systemctl stop docker
  2. systemctl start docker

Anything else we need to know?:

Environment:

{
  "name": "k8s-pod-network",
  "cniVersion": "0.3.1",
  "plugins": [
    {
      "type": "calico",
      "log_level": "info",
      "log_file_path": "/var/log/calico/cni/cni.log",
      "datastore_type": "kubernetes",
      "nodename": "master1",
      "mtu": 0,
      "ipam": {
          "type": "calico-ipam"
      },
      "policy": {
          "type": "k8s"
      },
      "kubernetes": {
          "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
      }
    },
    {
      "type": "portmap",
      "snat": true,
      "capabilities": {"portMappings": true}
    },
    {
      "type": "bandwidth",
      "capabilities": {"bandwidth": true}
    }
  ]
}
dougbtv commented 1 month ago

Multus doesn't actually create any iptables rules itself, other delegate CNI plugins may, likely calico in this case.