explore how to align EVPN GW to cloud APIs

[glimchb]

How I started with Cloud APIs using 4 objects:

• VPC (aka vrf object): https://github.com/opiproject/opi-api/blob/main/network/cloud/v1alpha1/vpc.proto
• Subnet (aka SVI/LogicalBridge object): https://github.com/opiproject/opi-api/blob/main/network/cloud/v1alpha1/subnet.proto
• Tunnel (for encapsulating packets): https://github.com/opiproject/opi-api/blob/main/network/cloud/v1alpha1/tunnel.proto
• Interface (Bridge Port object): https://github.com/opiproject/opi-api/blob/main/network/cloud/v1alpha1/interface.proto (can add vlan-id to HostIfSpec)

Now need to map all that work to EvpnGW APIs

• https://github.com/opiproject/opi-api/blob/main/network/evpn-gw/v1alpha1/l2_xpu_infra_mgr.proto
• https://github.com/opiproject/opi-api/blob/main/network/evpn-gw/v1alpha1/l3_xpu_infra_mgr.proto

[glimchb]

Example from @mardim91 to help transition the cloud APIs to evpn APIs:

Assumptions

• whatever is in SPEC of the object we care and need to configure
• whatever is in STATUS of the object we can auto-generate and return to caller
  • example: mac on the bridge of vrf table id

High level

• L3VPN use case:

  # generate <routing-table-number> in go code and return in reply
  ip link add <vrf-name> type vrf table <routing-table-number>
  ip link set <vrf-name> up
  ip address add <vrf-loopback> dev <vrf-name>
  # Add low-prio default route. Otherwise a miss leads to lookup in the next higher table
  ip route add throw default table <routing-table-number> proto evpn-gw-br metric 9999
  # you can generate a default rmac through go code if you want. Not big deal
  ip link add br-<vrf-name> address <rmac> type bridge
  ip link set br-<vrf-name> master <vrf-name> up
  ip link add vxlan-<vrf-name> type vxlan id <VRF-vni> local <Vtep-ip> dstport 4789 nolearning proxy
  ip link set vxlan-<vrf-name> master br-<vrf-name> up
  vtysh \
  -c "conf t" \
  -c "vrf <vrf-name>" \
  -c "  vni <vni>" \
  -c "  exit-vrf" \
  -c "exit"
  conf t" \
  -c "router bgp 65000 vrf <vrf-name>" \
  -c "  bgp router-id <vrf-loopback>" \
  -c "  no bgp ebgp-requires-policy" \
  -c "  no bgp hard-administrative-reset" \
  -c "  no bgp graceful-restart notification" \
  -c "  address-family ipv4 unicast" \
  -c "    redistribute connected" \
  -c "    redistribute static" \
  -c "    exit-address-family" \
  -c "  address-family l2vpn evpn" \
  -c "    advertise ipv4 unicast" \
  -c "    exit-address-family" \
  -c "exit"

• L2VPN use case:

  ip link add br-tenant type bridge vlan_default_pvid 0 vlan_filtering 1 vlan_protocol 802.1Q
  ip link set br-tenant up
  ip link add vxlan-<LB-vlan-id> type vxlan id <LB-vni> local <vtep-ip> dstport 4789 nolearning proxy
  ip link set vxlan-<LB-vlan-id> master br-tenant up
  bridge vlan add dev vxlan-<LB-vlan-id> vid <LB-vlan-id> pvid untagged
  bridge link set dev vxlan-<LB-vlan-id> neigh_suppress on

  the br-tenant is a common vlan enabled bridge for all the L2VPNs that you will create that means that every time you create a new L2VPN you only need to create a vlan on the br-tenant that matches the VNI of he L2VPN instance Now if you want to connect a Logical Bridge (L2VPN enabled or not) with a Routing L3VNI instance you need to create an SVI interface

More details

• init of the bridge (called once, not in any API):

  ip link add br-tenant type bridge vlan_default_pvid 0 vlan_filtering 1 vlan_protocol 802.1Q
  ip link set br-tenant up

• in CreateVRF call:

  # generate <routing-table-number> in go code and return in reply
  ip link add <vrf-name> type vrf table <routing-table-number>
  ip link set <vrf-name> up
  ip address add <vrf-loopback> dev <vrf-name>
  # Add low-prio default route. Otherwise a miss leads to lookup in the next higher table
  ip route add throw default table <routing-table-number> proto evpn-gw-br metric 9999
  # you can generate a default rmac through go code if you want. Not big deal
  ip link add br-<vrf-name> address <rmac> type bridge
  ip link set br-<vrf-name> master <vrf-name> up
  ip link add vxlan-<vrf-name> type vxlan id <VRF-vni> local <Vtep-ip> dstport 4789 nolearning proxy
  ip link set vxlan-<vrf-name> master br-<vrf-name> up

• in CreateLogicalBridge call:

  ip link add vxlan-<LB-vlan-id> type vxlan id <LB-vni> local <vtep-ip> dstport 4789 nolearning proxy
  ip link set vxlan-<LB-vlan-id> master br-tenant up
  bridge vlan add dev vxlan-<LB-vlan-id> vid <LB-vlan-id> pvid untagged
  bridge link set dev vxlan-<LB-vlan-id> neigh_suppress on

• in CreateSvi call:

  link_svi = <vrf-name>-<vlan-id>
  # Allow the VLAN as tag on the br-tenant bridge interface
  bridge vlan add dev br-tenant vid <vlan-id> self
  #Create a VLAN sub-interface on br-tenant as SVI interface
  ip link add link br-tenant name <link_svi> type vlan id <vlan-id>
  # Assign the GW MAC address to the SVI interface
  ip link set <link_svi> address <svi-mac>
  # Enslave the SVI interface to the VRF
  ip link set <link_svi> master <vrf-name> up
  # Learn neighbors from gratuitous ARPs
  sysctl -w net.ipv4.conf.<link_svi>.arp_accept=1
  # Assign the GW IP addresses to the SVI interface
  ip address add <svi-ip-with prefixlength> dev <link_svi>
  vtysh \
      -c "conf t" \
      -c "router bgp 65000 vrf <vrf-name>" \
      -c "  bgp disable-ebgp-connected-route-check" \
      -c "  neighbor <link-svi> peer-group" \
      -c "  neighbor <link-svi> remote-as <remote_as>" \
      -c "  neighbor <link-svi> update-source <svi-ip>" \
      -c "  neighbor {link_svi} as-override" \
      -c "  neighbor {link_svi} soft-reconfiguration inbound" \
      -c "  bgp listen range <svi-ip-with prefixlength> peer-group {link_svi}" \
      -c "exit"

• we can bother for bridge ports later...