Support other network plugins (CNI) than canal in MKS

[arcalys]

Add support for multiple network plugins (e.g. calico, cilium, weave,...) and let the user choose/define it at cluster creation through the GUI, CLI and Terraform.

[mhurtrel]

Hello @arcalys and thanks for your feature request. We have been exploring this idea, but currently lack a substantial and motivated demand count for a given other CNI to make sure this is worth maintaining multiple CNI for each future version of Kubernetes.

Could you share details about the CNI you wish to use and the use case it would enable ?

[arcalys]

Hello @mhurtrel,

On our end, we have self-hosted clusters running with Cilium, which we like especially for its observability stack, encryption and a bunch of other features (multi-clusters mesh, security features and policies, DNS policies,...).

For users in general though, I'd say the transition/migration to MKS would be easier if one does not have to change everything and rewrite every existing resource (network policies,...).

[rverchere]

+1 for Cilium which add great features on observability and security sides!

[seb-835]

+1 for cilium, it add Layer 7 network Policies features.

[Arcahub]

+1 for cilium, the observability capacties with Hubble and tracing capacity, security with previously mentionned Layer 7 network Policies but also sidecar-less service mesh (and since last release e2e encryption), Tetragon and finally native Ingress and Gateway API support. To be 100% transparent my company is currently thinking of migrating from ovh to other cloud provider only to have cilium support since it's that much important for us.

If I can help with this feature, I would be glad to.

[theyough]

it adds Multi-region clustering

[romain-fluttaz]

Cilium will be a game changer for your current managed Kubernetes offer.

https://github.com/ovh/public-cloud-roadmap/issues/116 helped a lot, but it not the same as a CNI like cilium.

:eyes: at:

• https://docs.cilium.io/en/stable/network/egress-gateway/
• Hubble to debug ;-)
• With https://github.com/cilium/cilium/issues/20550 fixed in https://github.com/cilium/cilium/pull/27464 :kissing_heart:

:+1:

[Yayg]

Hi, is there a plan for integrating this feature?

This would be indeed a game changer since Cilium is the only CNI able to offer a FQDN based network policy. Hope this will be integrated soon.

[mhurtrel]

Hi @Yayg I confirm that we will work on integrating an alternative CNI, and Clilium is currently the one we consider. However it is not planned for the next 6 months, sorry.

Please note that in the meantime, Cilium is part of the CNIs supported in Managed Rancher Service (for self managed clusters with RKE). If it is an aleternative for you, do not hesitate to join the alpha (private beta) : https://labs.ovhcloud.com/en/managed-rancher-service/

[yadutaf]

For the record, I've been able to setup Cilium in chaining mode on MKS on a single-node PoC cluster with:

cilium install --version 1.14.6 \
    --set cni.chainingTarget=k8s-pod-network \
    --set cni.chainingMode=generic-veth \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=true \
    --set routingMode=native \
    --set enableIPv4Masquerade=false \
    --set enableIPv6Masquerade=false  \
    --set operator.replicas=1

The install survived a MKS minor version update and Hubble was correctly reporting L3/L4 streams.

However, I was not able to get L7 rules and observability to work (known issue). In particular, I need to filter egress streams based on the FQDN.

[yadutaf]

Alternatively, an option like Azure's "Bring your own CNI" would be great for us. If this mode is selected, OVH would of course no longer provide support for CNI related topics but the customer would retain the benefits of a the managed control plane and nodes.

[royolsen]

+1 for Cilium. It is an absolute requirement for us.

[Davidffry]

+1 for cilium !

[hbrombeer]

Any updates? @mhurtrel