search

sonic-net / SONiC

Landing page for Software for Open Networking in the Cloud (SONiC) - https://sonic-net.github.io/SONiC/
2.22k stars 1.12k forks source link

VxLAN configuration doesn't work on SONiC VS #688

Closed DukeOwlington closed 3 years ago

DukeOwlington commented 3 years ago

Hello, We are very interested in how the L3 VxLAN on SONiC is configured. We are using SONiC VS at the moment for the purpose of testing its capabilities. On the picture below you can see the scheme that we are using for VxLAN capabilities test: 1

When we're trying to set the next configuration on Leaf-1 and Leaf-2 machines:

Leaf-1 configuration

{
    "VXLAN_TUNNEL": {
        "vxlantun10": {
            "src_ip": "10.1.12.70",
            "dst_ip": "10.1.12.33"
        }
    },
    "VNET": {
        "Vnet_10": {
            "vxlan_tunnel": "vxlantun10",
            "vni": "10"
        }
    },
    "BGP_NEIGHBOR": {
        "10.1.12.33": {
            "asn": "65001", 
            "holdtime": "180", 
            "keepalive": "60", 
            "local_addr": "10.1.12.70", 
            "name": "ARISTA03T2", 
            "nhopself": 0, 
            "rrclient": 0
        }
    }, 
    "DEVICE_METADATA": {
        "localhost": {
            "bgp_asn": "65000", 
            "hostname": "sonic", 
            "hwsku": "Force10-S6000", 
            "mac": "52:54:00:12:34:56", 
            "platform": "x86_64-kvm_x86_64-r0", 
            "type": "LeafRouter"
        }
    }, 
    "DEVICE_NEIGHBOR": {}, 
    "LOOPBACK_INTERFACE": {
        "Loopback0|10.62.0.2/32": {}
    },
    "INTERFACE": {
        "Ethernet0|10.1.12.70/24": {},
        "Ethernet4|192.168.0.1/24": { 
            "vnet_name": "Vnet_10"
        }
    },
    "PORT": {
        "Ethernet0": {
            "admin_status": "up", 
            "alias": "fortyGigE0/0", 
            "index": "0", 
            "lanes": "25,26,27,28", 
            "mtu": "9100", 
            "speed": "40000"
        },
        "Ethernet4": {
            "admin_status": "up", 
            "alias": "fortyGigE0/4", 
            "index": "1", 
            "lanes": "29,30,31,32", 
            "mtu": "9100", 
            "speed": "40000"
        }, 
        "Ethernet8": {
            "admin_status": "up", 
            "alias": "fortyGigE0/8", 
            "index": "2", 
            "lanes": "33,34,35,36", 
            "mtu": "9100", 
            "speed": "40000"
        }, 
        "Ethernet12": {
            "admin_status": "up", 
            "alias": "fortyGigE0/12", 
            "index": "3", 
            "lanes": "37,38,39,40", 
            "mtu": "9100", 
            "speed": "40000"
        }, 
        "Ethernet16": {
            "admin_status": "up", 
            "alias": "fortyGigE0/16", 
            "index": "4", 
            "lanes": "45,46,47,48", 
            "mtu": "9100", 
            "speed": "40000"
        }
    }
}

Leaf-2 Configuration

{
    "VXLAN_TUNNEL": {
        "vxlantun10": {
            "src_ip": "10.1.12.33",
            "dst_ip": "10.1.12.70"
        }
    },
    "VNET": {
        "Vnet_10": {
            "vxlan_tunnel": "vxlantun10",
            "vni": "10"
        }
    },
    "BGP_NEIGHBOR": {
        "10.1.12.70": {
            "asn": "65000", 
            "holdtime": "180", 
            "keepalive": "60", 
            "local_addr": "10.1.12.33", 
            "name": "ARISTA03T2", 
            "nhopself": 0, 
            "rrclient": 0
        }
    }, 
    "DEVICE_METADATA": {
        "localhost": {
            "bgp_asn": "65001", 
            "hostname": "sonic", 
            "hwsku": "Force10-S6000", 
            "mac": "52:54:00:12:34:56", 
            "platform": "x86_64-kvm_x86_64-r0", 
            "type": "LeafRouter"
        }
    }, 
    "DEVICE_NEIGHBOR": {}, 
    "LOOPBACK_INTERFACE": {
        "Loopback0|10.62.0.3/32": {}
    },
    "INTERFACE": {
        "Ethernet0|10.1.12.33/24": {},
        "Ethernet4|192.168.1.1/24": { 
            "vnet_name": "Vnet_10"
        }
    },
    "PORT": {
        "Ethernet0": {
            "admin_status": "up", 
            "alias": "fortyGigE0/0", 
            "index": "0", 
            "lanes": "25,26,27,28", 
            "mtu": "9100", 
            "speed": "40000"
        },
       "Ethernet4": {
            "admin_status": "up", 
            "alias": "fortyGigE0/4", 
            "index": "1", 
            "lanes": "29,30,31,32", 
            "mtu": "9100", 
            "speed": "40000"
        }, 
        "Ethernet8": {
            "admin_status": "up", 
            "alias": "fortyGigE0/8", 
            "index": "2", 
            "lanes": "33,34,35,36", 
            "mtu": "9100", 
            "speed": "40000"
        }, 
        "Ethernet12": {
            "admin_status": "up", 
            "alias": "fortyGigE0/12", 
            "index": "3", 
            "lanes": "37,38,39,40", 
            "mtu": "9100", 
            "speed": "40000"
        }, 
        "Ethernet16": {
            "admin_status": "up", 
            "alias": "fortyGigE0/16", 
            "index": "4", 
            "lanes": "45,46,47,48", 
            "mtu": "9100", 
            "speed": "40000"
        }
    }
}

VxLAN encapsulation doesn't work.

Frames that are coming out from ipterm-1 to Ethernet4 interface (Leaf-1) do not go anywhere further. We expect encapsulated frames to come out from Ethernet0 interface.

Can you tell us please how to configure L3 VxLAN on SONiC VS correctly?

prsunny commented 3 years ago

The configuration that you provided is for static L3 Vxlan which has a limitation on VS. It was implemented for the real ASIC. However you can use BGP EVPN with VNET configuration to do kernel based encapsulation or replace Leaf with a switch that supports Vxlan

DukeOwlington commented 3 years ago

@prsunny Ok, that's sufficient answer. Thank you!

techno-owl commented 2 years ago

Anyone was able to get this config working (statically configured tunnel)? I tried to simulate the encapped packet and isolate if the decap is working as expected or not. In my case, that's also not happening. We are not on VS, but on a switch which supports VXLAN.