Closed Levovar closed 3 years ago
Keeping it in WIP still, some polishing left to do - but the concept works!
Initial test NAD:
# cat example_nad.yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: ipvlan-conf
spec:
config: '{
"name": "vlantest",
"cniVersion": "0.3.1",
"type": "macvlan",
"capabilities": { "ips": true },
**"master": "tenant-bond",**
"vlan": 500,
"ipam": {
"type": "static",
"routes": [
{
"dst": "0.0.0.0/0",
"gw": "10.1.1.1"
}
]
}
}'
Interface created automatically on node when the network is created:
# ip l | grep vlantest
# kubectl create -f example_nad.yaml
networkattachmentdefinition.k8s.cni.cncf.io/ipvlan-conf created
# ip l | grep vlantest
568: vlantest.500@tenant-bond: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default
The correct device name is patched back into the network so the CNI connects child interfaces to the right parent:
# kubectl get network-attachment-definitions.k8s.cni.cncf.io ipvlan-conf -o yaml
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
creationTimestamp: "2021-03-01T17:21:11Z"
generation: 2
managedFields:
- apiVersion: k8s.cni.cncf.io/v1
fieldsType: FieldsV1
fieldsV1:
f:spec: {}
manager: kubectl-create
operation: Update
time: "2021-03-01T17:21:11Z"
- apiVersion: k8s.cni.cncf.io/v1
fieldsType: FieldsV1
fieldsV1:
f:spec:
f:config: {}
manager: netwatcher
operation: Update
time: "2021-03-01T17:21:11Z"
name: ipvlan-conf
namespace: default
resourceVersion: "13636666"
selfLink: /apis/k8s.cni.cncf.io/v1/namespaces/default/network-attachment-definitions/ipvlan-conf
uid: a536024f-6ae8-46c8-9ad0-fe9ecbe80baf
spec:
config: '{"capabilities":{"ips":true},"cniVersion":"0.3.1","ipam":{"routes":[{"dst":"0.0.0.0/0","gw":"10.1.1.1"}],"type":"static"},"master":"vlantest.500","name":"vlantest","type":"macvlan","vlan":500}'
Lastly, host interface is cleaned-up together with the network:
# kubectl delete -f example_nad.yaml
networkattachmentdefinition.k8s.cni.cncf.io "ipvlan-conf" deleted
# ip l | grep vlantest
@dougbtv you might find this add interesting. Let me know if you did! :)
@levovar, thanks for the highlight! I do find it interesting and awesome! Nice one!
On Wed, Mar 10, 2021, 9:47 AM Levente Kale @.***> wrote:
Merged #247 https://github.com/nokia/danm/pull/247 into master.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nokia/danm/pull/247#event-4436892018, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJX7PNSNY3ZQY5MWUAEQWLTC6A6ZANCNFSM4YIUVR3Q .
Keeping it in WIP still, some polishing left to do - but the concept works!
Initial test NAD:
# cat example_nad.yaml apiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: ipvlan-conf spec: config: '{ "name": "vlantest", "cniVersion": "0.3.1", "type": "macvlan", "capabilities": { "ips": true }, **"master": "tenant-bond",** "vlan": 500, "ipam": { "type": "static", "routes": [ { "dst": "0.0.0.0/0", "gw": "10.1.1.1" } ] } }'
Interface created automatically on node when the network is created:
# ip l | grep vlantest # kubectl create -f example_nad.yaml networkattachmentdefinition.k8s.cni.cncf.io/ipvlan-conf created # ip l | grep vlantest 568: vlantest.500@tenant-bond: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default
The correct device name is patched back into the network so the CNI connects child interfaces to the right parent:
# kubectl get network-attachment-definitions.k8s.cni.cncf.io ipvlan-conf -o yaml apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: creationTimestamp: "2021-03-01T17:21:11Z" generation: 2 managedFields: - apiVersion: k8s.cni.cncf.io/v1 fieldsType: FieldsV1 fieldsV1: f:spec: {} manager: kubectl-create operation: Update time: "2021-03-01T17:21:11Z" - apiVersion: k8s.cni.cncf.io/v1 fieldsType: FieldsV1 fieldsV1: f:spec: f:config: {} manager: netwatcher operation: Update time: "2021-03-01T17:21:11Z" name: ipvlan-conf namespace: default resourceVersion: "13636666" selfLink: /apis/k8s.cni.cncf.io/v1/namespaces/default/network-attachment-definitions/ipvlan-conf uid: a536024f-6ae8-46c8-9ad0-fe9ecbe80baf spec: config: '{"capabilities":{"ips":true},"cniVersion":"0.3.1","ipam":{"routes":[{"dst":"0.0.0.0/0","gw":"10.1.1.1"}],"type":"static"},"master":"vlantest.500","name":"vlantest","type":"macvlan","vlan":500}'
Lastly, host interface is cleaned-up together with the network:
# kubectl delete -f example_nad.yaml networkattachmentdefinition.k8s.cni.cncf.io "ipvlan-conf" deleted # ip l | grep vlantest
metadata: name: ipvlan-conf .... "type": "macvlan",
mismatch..
In an effort to expand the existing netwatcher to be a more generic "Network Management Operator", this commit enhances it to be able to "speak" NAD, in addition to currently existing the 3 DANM APIs.
The idea is pretty simple: the Controller component interprets the CNI config inside NAD.Spec.Config same way as the CNI config in Spec.Options of any DANM network, then call the existing VLAN, and VxLAN management functionalities regardless which API triggered the Operator. As a result, MACVLAN and IPVLAN type NADs can now also enjoy the dynamic VLAN, and VxLAN host interface management functionalities previously exclusive to DANM API users.
Netwatcher also takes care of patching the name of the parent interface in Spec.Config, so the upstream CNIs connect the Pods to the right/newly created host interface.