networkop / meshnet-cni

a (K8s) CNI plugin to create arbitrary virtual network topologies
BSD 3-Clause "New" or "Revised" License
116 stars 27 forks source link

Problem while creating network interfaces #2

Closed Skazza94 closed 5 years ago

Skazza94 commented 5 years ago

Hi, I'm currently using your plugin into my project but I'm having issues with interfaces creation. In particular, sometimes there are situations where a particular eth can have more peers than just one. For example, look at this simple network:

immagine

And take r1 for example. It has 2 peers on eth0 with dns1 and user, and 2 peers on eth1 with r2 and r3. This is my json file for this device (with eth numbers increased by one ofc):

[
    {
        "local_intf": "eth1", 
        "local_ip": "10.0.0.1/24", 
        "peer_intf": "eth1", 
        "peer_ip": "10.0.0.2/24", 
        "peer_pod": "user", 
        "uuid": 916
    }, 
    {
        "local_intf": "eth1", 
        "local_ip": "10.0.0.1/24", 
        "peer_intf": "eth1", 
        "peer_ip": "10.0.0.3/24", 
        "peer_pod": "dns1", 
        "uuid": 246
    }, 
    {
        "local_intf": "eth2", 
        "local_ip": "1.0.0.1/16", 
        "peer_intf": "eth2", 
        "peer_ip": "1.0.0.2/16", 
        "peer_pod": "r2", 
        "uuid": 182
    }, 
    {
        "local_intf": "eth2", 
        "local_ip": "1.0.0.1/16", 
        "peer_intf": "eth2", 
        "peer_ip": "1.0.0.3/16", 
        "peer_pod": "r3", 
        "uuid": 318
    }
]

Sometimes i get this error during pod creation:

Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "367e71d2ab6d4c9df4fd053477506a5da3a0a2afc1a4fa240ace3a425da1c4e5" network for pod "r1": NetworkPlugin cni failed to set up pod "r1_default" network: failed to rename link kokoa34a2299 -> eth1: file exists

Or sometimes the pod runs but with non-deterministic interfaces created: immagine For example, eth1 is missing in the image.

Is it possibile to fix this issue? Also, can it be an "uuid" problem? What does it mean?

networkop commented 5 years ago

I think the issues you're seeing are expected, since meshnet-cni can only create point-to-point links. The idea is to simulate real physical topologies, which today are mostly p2p (except for wifi).

I thought about the possibility of creating multipoint links, which is what you're trying to achieve, and the easiest way would be to simulate multipoint connection by a simple L2 hub. so for example, R1, dns and user will be connected to another devices, let's call it H1, and this H1 will replicate packets received on different ports, turning this effectively into a shared multipoint link.

To simulate H1, you can try and build a container with linux bridge connected to all containers interfaces and mac learning disabled (e.g. bridge link set dev MYBRIDGE learning off)

BTW, the uuid attribute is just a unique link ID, it needs to match on both ends of the link. You may want to have a look at k8s-topo, which takes care of uuid assignment and provides an easier to understand high-level YAML api.

Skazza94 commented 5 years ago

I see. I've solved it using multus-cni, so I can handle multiple interfaces on the same pod but with multipoint feature. Thanks for the reply by the way!

networkop commented 5 years ago

cool. closing the issue