Query or specify OVS port name/number to container instance

[edsealing]

I've been testing ovs-cni with multus as a datapath network for NFVi, and using flannel as the kubernetes network (eth0).

When a container is connected to OVS, it gets assigned a random port name (e.g. veth04e72a6) and an incremented port number in OVS (expected behavior). Due to the ephemeral nature of containers, it becomes difficult to manage the OVS rules to direct traffic to/from containers.

Would it be possible (or feasible) to query kubernetes or OVS to get the port name/number that is assigned to a specific container instance?

If not, is it feasible to modify ovs-cni to specify a OVS port # or port name that is permanently assigned to a specific container?

Thanks, and great work on this CNI.

[hwchiu]

Hi @edsealing. cc @John-Lin

We have store the mapping information between containerID and random port name under the /var/lib/cni/networks/$(bridge name). For example, if your ovs bridge name is br0 and the veth04e72a6 connect to a Pod called BusyBox (the containerID is 1234567890). You will find a file /var/lib/cni/networks/br0/1234567890 and its content is veth04e72a6.

I think you can implement a binary and deploy it as a daemon set, also mount the "/var/lib/cni/netwokrs" into that daemon and monitor the change of that directory.

The reason why we don't store the Pod name but containerID is that we get the containerID from the CNI standard interface. We need to find a way to get the pod name in the CNI procedure and we can store that information and we will start to see it.

Thanks.

[edsealing]

Hi @hwchiu. Had some time to look into this. I can see the mapping file getting created in /var/lib/cni/networks/br0/.

In my testing, I'm able to apply a Deployment and attach it to ovs-cni, through multus.

I'm retrieving the docker instance of the pods by using the following command: kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.containerStatuses[0].containerID}{"\n"}' | sed 's/docker:\/\///'

This provides me the output of:

netdiag-ds-h7gpd d8ecc26762b08d13e98611c1652351baa46e9c2f81ac6f269817add62afa9a47 netdiag-ds-n77xz a1ed24de3cf303fd99bab5e62df5a6ad0fef6a7e8fc49dce925a66dcd95c2052

These are the 2 pods that are running and the associated containerID.

When I look in /var/lib/cni/networks/br0/, I see the following file: /var/lib/cni/networks/br0/e17d61c84691ac5b03b26dcc359c91f501bb2bbf475dcb1468e16a31423d53dd The file listed isn't associated with any of the containerIDs that I have.

Digging into docker, I can see this containerID is associated with a pause-amd64:3.0 container.

CONTAINER ID	IMAGE	NAME
gcr.io/google_containers/pause-amd64:3.0	"/pause"	k8s_POD_netdiag-ds-n77xz_default_6823a2cf-f118-11e7-bafb-000c296efff2_0

When using Deployments, DaemonSets, etc is there any better way to get the correct container, pod, or info?

Cheers!

[hwchiu]

Hi @edsealing In the kubernetes, it create a pause container to hold a network for POD and all containers in the same POD will attach its network to pause container via network=xxxxxx(pause container ID). Have you tried to use the docker inspect to see all information about the container ID you got before? I will try to figure out the way to find the pause container ID from the pod and I will let you know after I find it.