Dualstack BGP does not work

[gurubert]

Hi,

I am trying to setup microcloud with OVN networking and BGP to the outside world. I get the connections to the external BGP routers but the updates sent by the LXD bgp daemon do not have the correct Ip family set.

If core.bgp_address is an IPv4 address (A.B.C.D) the IPv6 routes are updated with a nexthop of ::ffff:A.B.C.D which the receiving BGP routers reject as wrong family.

If core.bgp_address is an IPv6 address the IPv4 routes are updated with this IPv6 address as nexthop, which again the receiving BGP routers reject as wrong family.

How do i configure the LXD bgp daemon to talk IPv4 and IPv6 in parallel?

[roosterfish]

Hi @gurubert, can you please provide some concrete reproducer steps?

[gurubert]

This is the current configuration:

core.bgp_address: 198.51.100.91:179
core.bgp_asn: "64513"
core.bgp_routerid: 192.0.2.91

There are three nodes with end IPs of 91, 92 and 93.

# ip -br a s
lo               UNKNOWN        127.0.0.1/8 ::1/128 
eth0             UP             192.0.2.91/22 fe80::f43b:85ff:fee2:2e84/64 
eth1             UP             198.51.100.91/24 2001:DB8:2050:8201:2:91:0:1/64 fe80::ac81:f8ff:fed7:5266/64 
eth2             DOWN           
ovs-system       DOWN           
genev_sys_6081   UNKNOWN        fe80::44a5:4aff:fe08:f6be/64 
br-int           DOWN           
lxdfan0          UP             240.8.91.1/8 
lxdfan0-mtu      UNKNOWN        fe80::e4dd:63ff:fe67:ac0d/64 
lxdfan0-fan      UNKNOWN        fe80::2416:beff:fe29:443d/64 

The physical network definition for OVN on eth2 looks like this:

# lxc network show microcloud-test-transfer
config:
  bgp.peers.microcloudtestrtr01.address: 198.51.100.1
  bgp.peers.microcloudtestrtr01.asn: "64512"
  bgp.peers.microcloudtestrtr02.address: 198.51.100.2
  bgp.peers.microcloudtestrtr02.asn: "64512"
  dns.nameservers: 9.9.9.9
  ipv4.ovn.ranges: 198.51.100.128-198.51.100.254
  ipv4.routes: 203.0.113.0/24
  ipv6.ovn.ranges: 2001:DB8:2050:8201:3::1-2001:DB8:2050:8201:3:ffff:ffff:ffff
  ipv6.routes: 2001:DB8:2050:b003::/64
  volatile.last_state.created: "false"
description: ""
name: microcloud-test-transfer
type: physical
used_by: []
managed: true
status: Created
locations:
- microcloud-test01
- microcloud-test03
- microcloud-test02

I can then run this command to create an OVN network:

# lxc network create priv-microcloud-test01 --type=ovn network=microcloud-test-transfer ipv4.address=203.0.113.1/24 ipv4.nat=false ipv6.nat=false ipv6.address=2001:DB8:2050:b003::1/64

The BGP part of the BIRD configuration on the two routers looks like this:

template bgp t_ibgp {
    ipv4 {
        import all;
        export all;
    };
    ipv6 {
        next hop prefer global;
        import all;
        export all;
    };
}
protocol bgp microcloudtest01 from t_ibgp {
    local 198.51.100.1 as 64512;
    neighbor 198.51.100.91 as 64513;
}
protocol bgp microcloudtest02 from t_ibgp {
    local 198.51.100.1 as 64512;
    neighbor 198.51.100.92 as 64513;
}
protocol bgp microcloudtest03 from t_ibgp {
    local 198.51.100.1 as 64512;
    neighbor 198.51.100.93 as 64513;
}

I can switch the IPv4 addresses from 198.51.100.0/24 to IPv6 addresses from 2001:DB8:2050:8201::/64. I still get announcements for both IPv4 (203.0.113.0/24) and IPv6 (2001:DB8:2050:b003::/64) prefixes but one of them with the wrong family of the nexthop address:

Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest01.ipv4 > added [best] 203.0.113.0/24 0L 8G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: kernel_4.ipv4 < added 203.0.113.0/24 0L 8G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest02.ipv4 < added 203.0.113.0/24 0L 8G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest03.ipv4 < added 203.0.113.0/24 0L 8G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest01.ipv4 < rejected by protocol 203.0.113.0/24 0L 8G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest01: Invalid NEXT_HOP attribute - mismatched address family (198.51.100.91 for ipv6)
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest01: Invalid route 2001:db8:2050:b003::/64 withdrawn
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest02.ipv4 > added 203.0.113.0/24 0L 6G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: kernel_4.ipv4 < replaced 203.0.113.0/24 0L 8G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest02: Invalid NEXT_HOP attribute - mismatched address family (198.51.100.92 for ipv6)
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest02: Invalid route 2001:db8:2050:b003::/64 withdrawn
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest03.ipv4 > added 203.0.113.0/24 0L 7G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: kernel_4.ipv4 < replaced 203.0.113.0/24 0L 8G unicast
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest03: Invalid NEXT_HOP attribute - mismatched address family (198.51.100.93 for ipv6)
Nov 19 21:58:58 microcloud-test-rtr01 bird[102130]: microcloudtest03: Invalid route 2001:db8:2050:b003::/64 withdrawn

How would the lxd BGP process determine the IPv6 nexthop address when core.bgp_address is IPv4 or vice versa?

[tomponline]

I've moved this over to LXD repo as it is specific to LXD and not MicroCloud.