Open rookrunner opened 1 month ago
I tried using whereabout for this. And, now I am able to assign different IP's to the replicaset.
But, facing another issue. When I am doing deployment IP's were properly allocated to the replicas of the statefulset. But, when I tried upgrade looks like the IP's are not getting released by multus:
Warning FailedCreatePodSandBox <invalid> kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0889f8fc31b1fa54ee3a89d8864e7e943c8a93a952c6451b3c3ad5feb56f586f": [testnamespace/pod-1:multustestnamespace2]: error adding container to network "multustestnamespace2": Error at storage engine: Could not allocate IP in range: ip: 10.93.97.12 / - 10.93.97.13 / range: net.IPNet{IP:net.IP{0xa, 0x5d, 0x60, 0x0}, Mask:net.IPMask{0xff, 0xff, 0xf0, 0x0}}
Because of this my pod is not getting the IP and it remains in ContainerCreating state.
I have found this ticket which suggests to use cniVersion: 0.4.0. I am already using cniVersion: 0.4.0. Still facing the issue: https://github.com/k8snetworkplumbingwg/multus-cni/issues/738
This is my net-attach-def:
apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: annotations: meta.helm.sh/release-name: multus meta.helm.sh/release-namespace: testnamespace creationTimestamp: "2024-07-26T12:44:11Z" generation: 2 labels: app.kubernetes.io/managed-by: Helm name: multustest namespace: testnamespace resourceVersion: "493917757" uid: 9779f5ab-e969-4e04-9368-5d022746093c spec: config: |- { "plugins": [ { "cniVersion": "0.4.0", "type": "ipvlan", "master": "eth1", "mode": "l2", "ipam": { "type": "whereabouts", "range": "10.93.96.0/20", "range_start": "10.93.97.12", "range_end": "10.93.97.13", "log_file" : "/tmp/whereabouts.log", "log_level" : "error" } }, { "type": "sbr" } ] }
To be noted I have only 2 IP's in the range(10.93.97.12 - 10.93.97.13). Also, when I dont use sbr, it seems to be working. So, IP not releasing problem only exists with "sbr"
Could anyone please help
@dougbtv Could you please help here. Is this a known issue with chained CNI plugins.
An update: This config has worked. The only difference being "cniVersion" outside of plugins. Seems "cniVersion" is part of both "plugins" as well as "config" structs. I had put it in wrong place.
config: '{ "cniVersion": "0.4.0", "plugins": [ { "type": "ipvlan", "master": "eth1",
"mode": "l2", "ipam": { "type": "whereabouts", "range": "10.93.96.0/20", "range_start":
"10.93.97.12", "range_end": "10.93.97.13" } },{"type": "sbr"}] }'
Thanks!
I have 2 NetworkAttachmentDefinition with same definition like below, only IP are changed: