This repository contains the Golang source code for the Megalos CNI Plugin and the Megalos BGP Manager. Megalos CNI is compatible with Kubernetes v1.25+. Previous versions are not supported.
This plugin creates pure L2 LANs distributed across different worker nodes using VXLAN.
Before using the kathara-daemonset
, the Multus CNI must be deployed in the cluster.
See the official installation guide.
Once you have deployed the Multus CNI, you can deploy the kathara-daemonset
simply typing:
kubectl apply -f https://raw.githubusercontent.com/KatharaFramework/Megalos-CNI/master/kathara-daemonset.yml
Beware: Megalos CNI is used only for additional Pod interfaces created by Multus CNI! For the eth0
interface (required by Kubernetes) you must leverage on another CNI that manages L3 (e.g. Flannel, Calico).
NOTE: currently, Megalos CNI does not work on Amazon Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS) and ready-to-go cloud instances of Kubernetes, it only works on self-hosted clusters (since iptables
needs to install some custom rules).
This CNI creates a VXLAN network overlay over the Kubernetes cluster network. By default, VXLAN uses L3 multicast groups to deliver BUM traffic and to learn remote MAC Addresses, but multicast traffic is not always permitted inside a Kubernetes cluster network (see public clouds). To overcome this limitation, we disable the default VXLAN MAC Learning and we replace it with EVPN BGP, which ensures unicast traffic.
The BGP Manager works as follows:
When a Pod is created, the CNI works as follows:
veth
pairs.At this point, the BGP control plane is able to fetch the information of Pod network interfaces. So, MAC Addresses of Pod interfaces are announced over BGP to the Route Reflector and reflected to all the other worker BGP speakers. Each worker BGP Pod receives the announce: if a VTEP for a specific VNI announced in the BGP message is present, the VTEP saves the association between MAC Address and the IP Address of the worker node where the new interfaces of that VNI are deployed.
In this repository you'll find two folders:
megalos-cni
: Golang CNI source code for Megalos.bgp-manager
: Dockerfile and Bash scripts to create the kathara/megalos-bgp-manager
Docker Image.IMAGE_NAME
variable in Makefile
with a custom tag <CUSTOM_NAME>
.make all
, this will:
amd64
and arm64
architectures.kathara-daemonset.yml
file and replace kathara/megalos-bgp-manager
with <CUSTOM_NAME>
.kubeadm create -f kathara-daemonset.yml
.It is possible to leverage on Megalos CNI as a standalone Multus CNI, in order to create pure L2 networks.
After following the Usage section, we can deploy an example network contained in a network.yaml
file:
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: test-network-1
spec:
config: '{
"cniVersion": "0.3.0",
"name": "net1",
"type": "megalos",
"suffix": "ntwork",
"vxlanId": 10
}'
The config
JSON contains the CNI configuration. The values are:
name
: a name for the network (maximum 4 characters)suffix
: used by Kathará to distinguish between networks with the same name
of different users (6 characters)vxlanId
: Megalos CNI relies on VXLAN, so you should assign a VXLAN ID manually (Kathará does it automatically)At this point, we can create the network with the following command:
kubectl apply -f network.yaml
This configuration will create a VTEP, with VXLAN ID = 10, associated to the network net1-ntwork
.