search

k8snetworkplumbingwg / multus-cni

A CNI meta-plugin for multi-homed pods in Kubernetes
Apache License 2.0
2.27k stars 577 forks source link

Failures with read only host /run path #1260

Open rothgar opened 2 months ago

rothgar commented 2 months ago

What would you like to be added: On some Linux distros (especially ones with read only/immutable file systems) the /run/netns path may not be writable. This is used by the daemonset and causes an error.

This causes an unintuitive error in the daemonset when trying to run a pod

error adding container to network "cbr0": DelegateAdd:
 cannot set "" interface name to "eth0": validateIfName: no net namespace /var/run/netns/cni-e05e22ed-dadf-c935-5239-7eb8b4e13169 found: failed to
 Statfs "/var/run/netns/cni-e05e22ed-dadf-c935-5239-7eb8b4e13169": no such file or directory

because the daemonset is not able to write to that path.

Why is this needed: By changing the mount point to /var/run/netns or a similar writable path the daemonset works. Is there a way we could build this logic into the process to verify if it's able to write to that folder and/or mount both paths and verify which path should be written.

I tested this on Talos Linux and changing the mount works.

I'm open to other ideas on how to make this seamless for users.

I don't know for sure but it's possible this also affects Amazon Bottlerocket, Fedora CoreOS, Flatcar Linux, Elemental (Rancher), Kairos. All of them have some form of read only/immutable root file systems.

dougbtv commented 1 month ago

Our advice generally is to modify https://github.com/k8snetworkplumbingwg/multus-cni/blob/master/deployments/multus-daemonset-thick.yml#L246 to your liking.

There's a potential to provide another example deployment, or documentation. Up for ideas as well

rothgar commented 1 month ago

What would be the preferred way of adding examples for this?

  1. kustomize overlay with additional documentation - eg deployments/talos/{kustomize.yaml, README.md}
  2. duplicate thick yaml and modify the values in place and add a name the file something like multis-daemonset-thick-talos.yaml
  3. helm chart? I don't think a helm chart is provided so I can't modify variables there

I think the first option would probably be the cleanest and easiest to maintain.