search

FRRouting / frr

The FRRouting Protocol Suite
https://frrouting.org/
Other
3.33k stars 1.25k forks source link

Failed to build evpn and vxlan distributed symmetric gateway on frr version 7.4 #6785

Closed liuxc0116 closed 4 years ago

liuxc0116 commented 4 years ago

Describe the bug Failed to build evpn and vxlan distributed symmetric gateway on frr version 7.4

Topology image

The configuration of R1 is as follows Configure interface

#!/bin/bash

#enable forward
sysctl -w net.ipv4.ip_forward=1  
sysctl -p

#add host1
ip netns add host1
ip link add veth1 type veth peer name eth0 netns host1
ip netns exec host1 ip link set lo up
ip netns exec host1 ip link set eth0 up
ip netns exec host1 ip addr add 2.2.2.2/24 dev eth0
ip netns exec host1 ip route add default via 2.2.2.254 dev eth0

#add l2vni
ip link add br10 type bridge
ip link add vxlan10 type vxlan id 10 local 172.16.3.245 dstport 4789 nolearning
ip link set br10 up
ip link set veth1 up
ip link set vxlan10 up
ip link set veth1 master br10
ip link set vxlan10 master br10
ip link set dev br10 address 00:00:01:02:03:10
ip addr add 2.2.2.254/24 dev br10

#add l3 vni
ip link add br100 type bridge
ip link add vxlan100 type vxlan id 100 local 172.16.3.245 dstport 4789 nolearning
ip link set br100 up
ip link set vxlan100 up
ip link set vxlan100 master br100  
ip link set dev br100 address 00:00:01:02:03:04

# add vrf
ip link add evpn-vrf type vrf table 100
ip link set evpn-vrf up
ip link set br100 master evpn-vrf
ip link set br10 master evpn-vrf

Configure frr

vrf evpn-vrf
 vni 100
 exit-vrf
!
router bgp 7675
 bgp router-id 172.16.3.245
 neighbor 172.16.3.246 remote-as 7675
 neighbor 172.16.3.246 update-source 172.16.3.245
 neighbor 172.16.3.246 timers connect 10
 !
 address-family l2vpn evpn
  neighbor 172.16.3.246 activate
  neighbor 172.16.3.246 route-reflector-client
  advertise-all-vni
 exit-address-family
!
router bgp 7675 vrf evpn-vrf
 !
 address-family ipv4 unicast
  redistribute connected
  redistribute static
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!

The configuration of R2 is as follows Configure interface

#!/bin/bash

#enable forward
sysctl -w net.ipv4.ip_forward=1  
sysctl -p

#add host2
ip netns add host2
ip link add veth2 type veth peer name eth0 netns host2
ip netns exec host2 ip link set lo up
ip netns exec host2 ip link set eth0 up
ip netns exec host2 ip addr add 1.1.1.1/24 dev eth0
ip netns exec host2 ip route add default via 1.1.1.254 dev eth0 

#add br20
ip link add br20 type bridge
ip link set br20 up
ip link set veth2 up
ip link set veth2 master br20
ip addr add 1.1.1.254/24 dev br20

#add host3
ip netns add host3
ip link add veth3 type veth peer name eth0 netns host3
ip netns exec host3 ip link set lo up
ip netns exec host3 ip link set eth0 up
ip netns exec host3 ip addr add 2.2.2.3/24 dev eth0
ip netns exec host3 ip route add default via 2.2.2.254 dev eth0

ip link add br10 type bridge
ip link add vxlan10 type vxlan id 10 local 172.16.3.246 dstport 4789 nolearning
ip link set vxlan10 up
ip link set br10 up
ip link set veth3 up
ip link set veth3 master br10
ip link set vxlan10 master br10
ip addr add 2.2.2.254/24 dev br10
ip link set dev br10 address 00:00:01:02:03:10

#add l3vni
ip link add br100 type bridge
ip link add vxlan100 type vxlan id 100 local 172.16.3.246 dstport 4789 nolearning
ip link set br100 up
ip link set vxlan100 up
ip link set vxlan100 master br100  
ip link set dev br100 address 00:00:01:02:03:05 

#add vrf
ip link add evpn-vrf type vrf table 100
ip link set evpn-vrf up
ip link set br100 master evpn-vrf  
ip link set br20 master evpn-vrf 
ip link set br10 master evpn-vrf

Configure frr

vrf evpn-vrf
 vni 100
 exit-vrf
!
router bgp 7675
 bgp router-id 172.16.3.246
 neighbor 172.16.3.245 remote-as 7675
 neighbor 172.16.3.245 update-source 172.16.3.246
 neighbor 172.16.3.245 timers connect 10
 !
 address-family l2vpn evpn
  neighbor 172.16.3.245 activate
  neighbor 172.16.3.245 route-reflector-client
  advertise-all-vni
 exit-address-family
!
router bgp 7675 vrf evpn-vrf
 !
 address-family ipv4 unicast
  redistribute connected
  redistribute static
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
 exit-address-family
!

After version 7.3, use the above configuration, host1 is unreachable host2 by ping

Under 7.2.1 version,host1 is reachable host2 by ping, bgp and routing information are as follows

R1

Linker# show ip bgp l2vpn evpn
BGP table version is 2, local router ID is 172.16.3.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i[5]:[0]:[24]:[1.1.1.0]
                    172.16.3.246             0    100      0 ?
                    RT:7675:100 ET:8 Rmac:00:00:01:02:03:05
*> [5]:[0]:[24]:[2.2.2.0]
                    172.16.3.245             0         32768 ?
                    ET:8 RT:7675:100 Rmac:00:00:01:02:03:04
*> [2]:[0]:[48]:[b2:de:3d:99:93:15]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10 RT:7675:100 Rmac:00:00:01:02:03:04
*> [2]:[0]:[48]:[b2:de:3d:99:93:15]:[32]:[2.2.2.2]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10 RT:7675:100 Rmac:00:00:01:02:03:04
*> [2]:[0]:[48]:[b2:de:3d:99:93:15]:[128]:[fe80::b0de:3dff:fe99:9315]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10
*> [3]:[0]:[32]:[172.16.3.245]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10
*>i[2]:[0]:[48]:[5a:33:e4:4a:e9:20]
                    172.16.3.246                  100      0 i
                    RT:7675:10 RT:7675:100 ET:8 Rmac:00:00:01:02:03:05
*>i[2]:[0]:[48]:[5a:33:e4:4a:e9:20]:[128]:[fe80::5833:e4ff:fe4a:e920]
                    172.16.3.246                  100      0 i
                    RT:7675:10 ET:8
*>i[3]:[0]:[32]:[172.16.3.246]
                    172.16.3.246                  100      0 i
                    RT:7675:10 ET:8

Displayed 9 out of 9 total prefixes

Linker# show ip route vrf evpn-vrf
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF evpn-vrf:
B>* 1.1.1.0/24 [200/0] via 172.16.3.246, br100 onlink, 00:13:35
C>* 2.2.2.0/24 is directly connected, br10, 00:13:35

R2

Linker# show ip bgp l2vpn evpn
BGP table version is 3, local router ID is 172.16.3.246
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> [5]:[0]:[24]:[1.1.1.0]
                    172.16.3.246             0         32768 ?
                    ET:8 RT:7675:100 Rmac:00:00:01:02:03:05
*>i[5]:[0]:[24]:[2.2.2.0]
                    172.16.3.245             0    100      0 ?
                    RT:7675:100 ET:8 Rmac:00:00:01:02:03:04
*  [5]:[0]:[24]:[2.2.2.0]
                    172.16.3.246             0         32768 ?
                    ET:8 RT:7675:100 Rmac:00:00:01:02:03:05
*>i[2]:[0]:[48]:[b2:de:3d:99:93:15]
                    172.16.3.245                  100      0 i
                    RT:7675:10 RT:7675:100 ET:8 Rmac:00:00:01:02:03:04
*>i[2]:[0]:[48]:[b2:de:3d:99:93:15]:[32]:[2.2.2.2]
                    172.16.3.245                  100      0 i
                    RT:7675:10 RT:7675:100 ET:8 Rmac:00:00:01:02:03:04
*>i[2]:[0]:[48]:[b2:de:3d:99:93:15]:[128]:[fe80::b0de:3dff:fe99:9315]
                    172.16.3.245                  100      0 i
                    RT:7675:10 ET:8
*>i[3]:[0]:[32]:[172.16.3.245]
                    172.16.3.245                  100      0 i
                    RT:7675:10 ET:8
*> [2]:[0]:[48]:[5a:33:e4:4a:e9:20]
                    172.16.3.246                       32768 i
                    ET:8 RT:7675:10 RT:7675:100 Rmac:00:00:01:02:03:05
*> [2]:[0]:[48]:[5a:33:e4:4a:e9:20]:[128]:[fe80::5833:e4ff:fe4a:e920]
                    172.16.3.246                       32768 i
                    ET:8 RT:7675:10
*> [3]:[0]:[32]:[172.16.3.246]
                    172.16.3.246                       32768 i
                    ET:8 RT:7675:10

Displayed 10 out of 10 total prefixes

Linker# show ip route vrf evpn-vrf
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF evpn-vrf:
C>* 1.1.1.0/24 is directly connected, br20, 00:14:26
C>* 2.2.2.0/24 is directly connected, br10, 00:14:26

Under 7.4 version,host1 is unreachable host2 by ping, bgp and routing information are as follows

R1

Linker# show ip bgp l2vpn evpn 
BGP table version is 2, local router ID is 172.16.3.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 2.2.2.254:2
*>i[5]:[0]:[24]:[1.1.1.0]
                    172.16.3.246             0    100      0 ?
                    RT:7675:100 ET:8 Rmac:00:00:01:02:03:05
*> [5]:[0]:[24]:[2.2.2.0]
                    172.16.3.245             0         32768 ?
                    ET:8 RT:7675:100 Rmac:00:00:01:02:03:04
Route Distinguisher: 172.16.3.245:3
*> [2]:[0]:[48]:[4e:0d:dd:76:82:7e]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10 RT:7675:100 Rmac:00:00:01:02:03:04
*> [2]:[0]:[48]:[4e:0d:dd:76:82:7e]:[32]:[2.2.2.2]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10 RT:7675:100 Rmac:00:00:01:02:03:04
*> [2]:[0]:[48]:[4e:0d:dd:76:82:7e]:[128]:[fe80::4c0d:ddff:fe76:827e]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10
*> [3]:[0]:[32]:[172.16.3.245]
                    172.16.3.245                       32768 i
                    ET:8 RT:7675:10
Route Distinguisher: 172.16.3.246:3
*>i[2]:[0]:[48]:[9a:38:bb:7b:8d:70]
                    172.16.3.246                  100      0 i
                    RT:7675:10 RT:7675:100 ET:8 Rmac:00:00:01:02:03:05
*>i[2]:[0]:[48]:[9a:38:bb:7b:8d:70]:[128]:[fe80::9838:bbff:fe7b:8d70]
                    172.16.3.246                  100      0 i
                    RT:7675:10 ET:8
*>i[3]:[0]:[32]:[172.16.3.246]
                    172.16.3.246                  100      0 i
                    RT:7675:10 ET:8

Displayed 9 out of 9 total prefixes

Linker# show ip route vrf evpn-vrf
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF evpn-vrf:
B>* 1.1.1.0/24 [200/0] via 172.16.3.246, br100 onlink, weight 1, 00:06:05
C>* 2.2.2.0/24 is directly connected, br10, 00:06:11

R2

Linker# show ip bgp l2vpn evpn 
BGP table version is 2, local router ID is 172.16.3.246
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 2.2.2.254:2
*> [5]:[0]:[24]:[1.1.1.0]
                    172.16.3.246             0         32768 ?
                    ET:8 RT:7675:100 Rmac:00:00:01:02:03:05
*  [5]:[0]:[24]:[2.2.2.0]
                    172.16.3.246             0         32768 ?
                    ET:8 RT:7675:100 Rmac:00:00:01:02:03:05
*>i                 172.16.3.245             0    100      0 ?
                    RT:7675:100 ET:8 Rmac:00:00:01:02:03:04
Route Distinguisher: 172.16.3.245:3
*>i[2]:[0]:[48]:[4e:0d:dd:76:82:7e]
                    172.16.3.245                  100      0 i
                    RT:7675:10 RT:7675:100 ET:8 Rmac:00:00:01:02:03:04
*>i[2]:[0]:[48]:[4e:0d:dd:76:82:7e]:[32]:[2.2.2.2]
                    172.16.3.245                  100      0 i
                    RT:7675:10 RT:7675:100 ET:8 Rmac:00:00:01:02:03:04
*>i[2]:[0]:[48]:[4e:0d:dd:76:82:7e]:[128]:[fe80::4c0d:ddff:fe76:827e]
                    172.16.3.245                  100      0 i
                    RT:7675:10 ET:8
*>i[3]:[0]:[32]:[172.16.3.245]
                    172.16.3.245                  100      0 i
                    RT:7675:10 ET:8
Route Distinguisher: 172.16.3.246:3
*> [2]:[0]:[48]:[9a:38:bb:7b:8d:70]
                    172.16.3.246                       32768 i
                    ET:8 RT:7675:10 RT:7675:100 Rmac:00:00:01:02:03:05
*> [2]:[0]:[48]:[9a:38:bb:7b:8d:70]:[128]:[fe80::9838:bbff:fe7b:8d70]
                    172.16.3.246                       32768 i
                    ET:8 RT:7675:10
*> [3]:[0]:[32]:[172.16.3.246]
                    172.16.3.246                       32768 i
                    ET:8 RT:7675:10

Displayed 9 out of 10 total prefixes

Linker# show ip route vrf evpn-vrf
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route

VRF evpn-vrf:
C>* 1.1.1.0/24 is directly connected, br20, 00:07:15
C>* 2.2.2.0/24 is directly connected, br10, 00:07:15
B>* 2.2.2.2/32 [200/0] via 172.16.3.245, br100 onlink, weight 1, 00:04:31

Versions

sworleys commented 4 years ago

Versions

  • OS Kernel: [ubuntu 18.04]
  • FRR Version [frr-3.0 frr-3.1 frr-4.0]

For kernel version, please use uname -a and post that

liuxc0116 commented 4 years ago

Versions

  • OS Kernel: [ubuntu 18.04]
  • FRR Version [frr-3.0 frr-3.1 frr-4.0]

For kernel version, please use uname -a and post that

ubuntu 18.04 Linux ubuntu 4.15.0-29-generic #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

sworleys commented 4 years ago

@xThaid thinks this might be fixed with https://github.com/FRRouting/frr/pull/6752

xThaid commented 4 years ago

@liuxc0116 Did you test it on master?

liuxc0116 commented 4 years ago

@liuxc0116 Did you test it on master?

I just tested it on master. it works.

eqvinox commented 4 years ago

May need to backport another fix to 7.4?

sworleys commented 4 years ago

@liuxc0116 can you see if these commits in this PR https://github.com/FRRouting/frr/pull/6419 applied on stable/7.4 fix it for you?

liuxc0116 commented 4 years ago

@liuxc0116 can you see if these commits in this PR #6419 applied on stable/7.4 fix it for you?

image

yes, use the patch(in this PR #6419), it works.