Got BGP Rejected at Routing Table (Show IPv6 Route) - SRv6 L3VPN

[iqraaswad]

---

Describe the bug

[ X] Did you check if this is a duplicate issue? [ X] Did you test it on the latest FRRouting/frr master branch?

To Reproduce

Router 1 :

Router Configuration :

Current configuration:
!
frr version 8.1
frr defaults traditional
hostname pi3
no ip forwarding
service integrated-vtysh-config
!
ipv6 route 2001:d30:8000:8000::5/128 2001:d30:8000:243::15
!
vrf vrf1
 vni 4001
exit-vrf
!
interface lo
 ipv6 address 2001:d30:8000:8000::3/128
 ipv6 router isis SRv6
 isis circuit-type level-1
 isis passive
exit
!
interface eth0.241
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
interface eth0.243
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
interface eth0.244
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
router bgp 30
 bgp router-id 3.3.3.3
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp bestpath as-path multipath-relax
 neighbor iPartner peer-group
 neighbor iPartner remote-as 30
 neighbor iPartner tcp-mss 1200
 neighbor iPartner update-source lo
 neighbor iPartner capability extended-nexthop
 neighbor 2001:d30:8000:8000::5 peer-group iPartner
 neighbor 2001:d30:8000:8000::5 tcp-mss 1200
 !
 segment-routing srv6
  locator SRv6_Loc
 exit
 !
 address-family ipv6 unicast
  maximum-paths 2
 exit-address-family
 !
 address-family ipv6 vpn
  neighbor iPartner activate
  neighbor iPartner route-reflector-client
 exit-address-family
exit
!
router bgp 30 vrf vrf1
 bgp router-id 3.3.3.3
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 !
 address-family ipv6 unicast
  network 2001:d30:8000:241::/64
  redistribute connected
  sid vpn export auto
  rd vpn export 30:131
  rt vpn both 30:13
  export vpn
  import vpn
 exit-address-family
exit
!
router isis SRv6
 is-type level-1
 net 49.0001.0001.0001.0003.00
 spf prefix-priority critical ACL_CRITICAL
 topology ipv6-unicast
 fast-reroute priority-limit critical level-1
 segment-routing on
 segment-routing node-msd 8
exit
!
ipv6 access-list ACL_CRITICAL seq 5 permit 2001:d30:8000:243::/64
ipv6 access-list ACL_CRITICAL seq 10 permit 2001:d30::/48
!
segment-routing
 srv6
  locators
   locator SRv6_Loc
    prefix 2001:d30:8000:3::/64
   exit
   !
  exit
  !
 exit
 !
 traffic-eng
 exit
exit
!
end

Router 2 :

Current configuration:
!
frr version 8.1
frr defaults traditional
hostname pi5
no ip forwarding
service integrated-vtysh-config
!
ipv6 route 2001:d30:8000:8000::3/128 2001:d30:8000:243::13
!
vrf vrf1
 vni 4001
exit-vrf
!
interface lo
 ipv6 address 2001:d30:8000:8000::5/128
 ipv6 router isis SRv6
 isis circuit-type level-1
 isis passive
exit
!
interface eth0.243
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
router bgp 30
 bgp router-id 5.5.5.5
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp bestpath as-path multipath-relax
 neighbor iPartner peer-group
 neighbor iPartner remote-as 30
 neighbor iPartner update-source lo
 neighbor iPartner capability extended-nexthop
 neighbor 2001:d30:8000:8000::3 peer-group iPartner
 !
 segment-routing srv6
  locator SRv6_Loc
 exit
 !
 address-family ipv6 vpn
  neighbor iPartner activate
 exit-address-family
exit
!
router bgp 30 vrf vrf1
 bgp router-id 5.5.5.5
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 !
 address-family ipv6 unicast
  redistribute connected
  sid vpn export auto
  rd vpn export 30:151
  rt vpn both 30:15
  export vpn
  import vpn
 exit-address-family
exit
!
router isis SRv6
 is-type level-1
 net 49.0001.0001.0001.0005.00
 lsp-mtu 1300
 spf prefix-priority critical ACL_CRITICAL
 topology ipv6-unicast
 fast-reroute priority-limit critical level-1
 segment-routing on
 segment-routing node-msd 8
exit
!
ipv6 access-list ACL_CRITICAL seq 5 permit 2001:d30:8000:243::/64
ipv6 access-list ACL_CRITICAL seq 10 permit 2001:d30::/48
!
segment-routing
 srv6
  locators
   locator SRv6_Loc
    prefix 2001:d30:8000:5::/64
   exit
   !
  exit
  !
 exit
 !
 traffic-eng
 exit
exit
!
end

Expected behavior

B> 2001:d30:8000:3:100::/128 [20/0] is directly connected, vrf1, seg6local End.DT6 table 10, seg6 ::, weight 1, 00:04:31

Screenshots

I got B>r 2001:d30:8000:3:100::/128 [20/0] is directly connected, vrf1, seg6local End.DT6 table 10, seg6 ::, weight 1, 00:04:31 Versions

• OS Version: Debian GNU/Linux 11 (bullseye)
• Kernel: Linux 5.10.63-v8+
• FRR Version: 8.1 stable version

Additional context

I got rejected status at routing table when using show ipv6 route command

[donaldsharp]

what does the log say about the rejection?

[iqraaswad]

I don't know how to produce a log. I have checked file /var/log/frr/frr.log but found nothing 🥲. I just got daemon status from that file

[iqraaswad]

what does the log say about the rejection?

I don't know exactly how to do that.. but this is my interface brief

Interface       Status  VRF             Addresses
---------       ------  ---             ---------
eth0            up      default         169.254.214.48/16
eth0.231        up      default         202.249.25.143/27
                                        + 2001:d30:101:2:b7bc:71c7:3ba1:badb/64
eth0.243        down    default         
eth0.244        up      default         192.168.244.13/24
                                        + 2001:d30:8000:244::13/64
lo              up      default         2001:d30:8000:8000::3/128
pimreg          up      default         
wlan0           down    default         

Interface       Status  VRF             Addresses
---------       ------  ---             ---------
eth0.241        up      vrf1            192.168.241.13/24
                                                  + 2001:d30:8000:241::13/64
pimreg10        up      vrf1            
vrf1                 up      vrf1            169.254.68.80/16

i create manually vrf1 using this command

 ip link add vrf1 type vrf table 10 
ip link set dev eth0.241 master vrf1

and when trying to ping eth0.241 from same router (FRRouting cli) ... i got

pi3# ping 2001:d30:8000:241::13
PING 2001:d30:8000:241::13(2001:d30:8000:241::13) 56 data bytes
From 2001:d30:8000:241::13 icmp_seq=1 Destination unreachable: Address unreachable
From 2001:d30:8000:241::13 icmp_seq=2 Destination unreachable: Address unreachable
From 2001:d30:8000:241::13 icmp_seq=3 Destination unreachable: Address unreachable
^C
--- 2001:d30:8000:241::13 ping statistics ---
5 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4078ms

any solutions please.. 😢

[donaldsharp]

log file /tmp/zebra.log debug zebra kernel debug zebra rib detail

then cause bgp to reinstall the route

[iqraaswad]

after do that... i got some error from zebra.log

2022/04/02 16:41:32 BGP: [KTTWK-0CPJ7][EC 100663303] sockopt_tcp_signature: setsockopt(24): Protocol not available
2022/04/02 16:41:32 BGP: [S3XWN-YG5FD][EC 33554494] Unable to set TCP MD5 option on socket for peer 2001:d30:8000:8000::4 (sock=24): Protocol not available
2022/04/02 16:41:32 ZEBRA: [S77YB-SX0AW] assign_srv6_locator_chunk: locator SRv6_Loc was not found
2022/04/02 16:41:32 ZEBRA: [XTSCQ-WA14G] Unable to assign locator chunk to bgp instance 0
2022/04/02 16:41:32 ZEBRA: [T1KCW-RSXEY] zebra_redistribute: client bgp 2001:d30:8000:240::/64(71) checking: selected=8, type=2, distance=0, metric=0 zebra_check_addr=1
2022/04/02 16:41:32 ZEBRA: [T1KCW-RSXEY] zebra_redistribute: client bgp fe80::/64(71) checking: selected=8, type=2, distance=0, metric=0 zebra_check_addr=0
2022/04/02 16:41:32 ZEBRA: [T1KCW-RSXEY] zebra_redistribute: client bgp fe80::/64(71) checking: selected=0, type=2, distance=0, metric=0 zebra_check_addr=0
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 ZEBRA: [V98V0-MTWPF] client 125 says hello and bids fair to announce only isis routes vrf=0
2022/04/02 16:41:32 ZEBRA: [TAFZ7-2C7QV] client protocol instance 1
2022/04/02 16:41:32 ZEBRA: [PNYQ5-RZEVA] rib_delete: (0:254):2001:d30:8000:10::/64: type 8 doesn't exist in rib
2022/04/02 16:41:32 ZEBRA: [PNYQ5-RZEVA] rib_delete: (0:254):2001:d30:8000:242::/64: type 8 doesn't exist in rib
2022/04/02 16:41:32 ZEBRA: [PNYQ5-RZEVA] rib_delete: (0:254):2001:d30:8000:8000::1/128: type 8 doesn't exist in rib
2022/04/02 16:41:32 ZEBRA: [MFYWV-KH3MC] rib_add_multipath_nhe: (0:?):2001:d30:8000:1:100::/128: Inserting route rn 0x559537ee80, re 0x5594c08520 (bgp) existing 0x0, same_count 0
2022/04/02 16:41:32 ZEBRA: [Q4T2G-E2SQF] rib_add_multipath_nhe: dumping RE entry 0x5594c08520 for 2001:d30:8000:1:100::/128 vrf default(0)
2022/04/02 16:41:32 ZEBRA: [M5M58-9PD2R] 2001:d30:8000:1:100::/128: uptime == 8831241, type == 9, instance == 0, table == 254
2022/04/02 16:41:32 ZEBRA: [RVZMM-N7DME] 2001:d30:8000:1:100::/128: metric == 0, mtu == 0, distance == 20, flags == Recursion status == None 
2022/04/02 16:41:32 ZEBRA: [Q1NW5-NWY7P] 2001:d30:8000:1:100::/128: nexthop_num == 1, nexthop_active_num == 0
2022/04/02 16:41:32 ZEBRA: [TFHQ8-TC30H] 2001:d30:8000:1:100::/128: NH vrf1[71] vrf default(0) wgt 1, with flags 
2022/04/02 16:41:32 ZEBRA: [SCETK-GQ9E4] 2001:d30:8000:1:100::/128: dump complete
2022/04/02 16:41:32 ZEBRA: [GCGMT-SQR82] rib_link: (0:?):2001:d30:8000:1:100::/128: rn 0x559537ee80 adding dest
2022/04/02 16:41:32 ZEBRA: [QEVVE-G3FQQ] rib_meta_queue_add: (0:254):2001:d30:8000:1:100::/128: queued rn 0x559537ee80 into sub-queue 6
2022/04/02 16:41:32 ZEBRA: [NZNZ4-7P54Y] default(0:254):2001:d30:8000:1:100::/128: Processing rn 0x559537ee80
2022/04/02 16:41:32 ZEBRA: [ZJVZ4-XEGPF] default(0:254):2001:d30:8000:1:100::/128: Examine re 0x5594c08520 (bgp) status: Changed flags: Recursion dist 20 metric 0
2022/04/02 16:41:32 ZEBRA: [JPJF4-TGCY5] default(0:254):2001:d30:8000:1:100::/128: After processing: old_selected 0x0 new_selected 0x5594c08520 old_fib 0x0 new_fib 0x5594c08520
2022/04/02 16:41:32 ZEBRA: [ZFRN1-EZNJZ] default(0:254):2001:d30:8000:1:100::/128: Adding route rn 0x559537ee80, re 0x5594c08520 (bgp)
2022/04/02 16:41:32 ZEBRA: [QZ1V6-CRT8D] default(0:254):2001:d30:8000:1:100::/128 rn 0x559537ee80 dequeued from sub-queue 6
2022/04/02 16:41:32 ZEBRA: [YXCJP-0WZWV] netlink_nexthop_msg_encode: ID (42): directly connected, vrf1(71) vrf default(0)  
2022/04/02 16:41:32 ZEBRA: [R43C6-KYHWT] netlink_nexthop_msg_encode: RTM_NEWNEXTHOP, id=42
2022/04/02 16:41:32 ZEBRA: [YXPF5-B2CE0] netlink_route_multipath_msg_encode: RTM_NEWROUTE 2001:d30:8000:1:100::/128 vrf 0(254)
2022/04/02 16:41:32 ZEBRA: [J87BH-XW5PP] netlink_route_multipath_msg_encode: 2001:d30:8000:1:100::/128 nhg_id is 42
2022/04/02 16:41:32 ZEBRA: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=132, msg cnt=2
2022/04/02 16:41:32 ZEBRA: [HSYZM-HV7HF] Extended Error: lwt encapsulation type not supported
2022/04/02 16:41:32 ZEBRA: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: Operation not supported, type=RTM_NEWNEXTHOP(104), seq=39, pid=2619130526
2022/04/02 16:41:32 ZEBRA: [QTT8V-3ZQ34] nl_batch_read_resp: netlink error message seq=39 
2022/04/02 16:41:32 ZEBRA: [HSYZM-HV7HF] Extended Error: Nexthop id does not exist
2022/04/02 16:41:32 ZEBRA: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: Invalid argument, type=RTM_NEWROUTE(24), seq=40, pid=2619130526
2022/04/02 16:41:32 ZEBRA: [QTT8V-3ZQ34] nl_batch_read_resp: netlink error message seq=40 
2022/04/02 16:41:32 ZEBRA: [P2XBZ-RAFQ5][EC 4043309074] Failed to install Nexthop ID (42) into the kernel
2022/04/02 16:41:32 ZEBRA: [TX7F6-F3TSG] default(0:254):2001:d30:8000:1:100::/128: Route install failed
2022/04/02 16:41:32 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 RIP: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 RIPNG: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 OSPF: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 OSPF6: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01

[donaldsharp]

2022/04/02 16:41:32 ZEBRA: [HSYZM-HV7HF] Extended Error: lwt encapsulation type not supported looks like your kernel does not support this encapsulation.

[donaldsharp]

@slankdev do you know what version of the linux kernel @iqraaswad should be using?

[iqraaswad]

thanks for mentioning kernel.. i've recompiling kernel.. and everything works now...

[ale0524]

你知道应该使用什么版本的 Linux 内核吗？

Hello, I also encountered the same problem on my end;

Operating system: Centos 7.6

Kernel version: 5.4.241-1. el7.elrepo.x86_ sixty-four

May I ask which kernel version should I use??

[ale0524]

thanks for mentioning kernel.. i've recompiling kernel.. and everything works now... Hello, I also encountered the same problem on my end;

Operating system: Centos 7.6

Kernel version: 5.4.241-1. el7.elrepo.x86_ sixty-four

May I ask which kernel version should I use??

[ale0524]

May 5 07:04:58 R1 zebra[5409]: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: Invalid argument, type=RTM_NEWROUTE(24), seq=6491, pid=4093804124 May 5 07:04:58 R1 zebra[5409]: [X5XE1-RS0SW][EC 4043309074] Failed to install Nexthop (392[fe80::4bfa:b4f7:bb81:6b1d if 3]) into the kernel May 5 07:04:58 R1 zebra[5409]: [VYKYC-709DP] evpn1(11:10):2001:db8:b::/64: Route install failed May 5 07:05:00 R1 pim6d[5440]: [PWY4E-BRD9V] zclient_read_nexthop: found non-ifindex nexthop type=6 for address *(default)

[ale0524]

May 5 06:23:52 R1 zebra[5409]: [SWQK6-6JY63][EC 4043309074] 0:254:2001:db8:2::/128: Failed to enqueue dataplane install

[ale0524]

[ale0524]

May 6 04:00:35 R1 pim6d[4083]: [PWY4E-BRD9V] zclient_read_nexthop: found non-ifindex nexthop type=6 for address (default) May 6 04:01:31 R2 pim6d[4064]: [PWY4E-BRD9V] zclient_read_nexthop: found non-ifindex nexthop type=6 for address (default)

[liuyuefengcn]

@iqraaswad Hello, I am also having the same issue, can you tell me what version of linux kernel I should use? Thank you so much! CentOS Linux release 8.5.2111 Linux version 4.18.0-348.el8.x86_64

[liuyuefengcn]

I've upgraded the linux kernel version to 6.5.7-1.el8.elrepo.x86_64, but the problem persists, can anyone help figure it out?

[ale0524]

@liuyuefengcn Currently, only these use cases are supported: IPv6 L3VPN (with SRv6 End.DT6) -> kernel >=4.14 -> config example here IPv4 L3VPN (with SRv6 End.DT4) -> kernel >= 5.11 -> config example here IPv4/IPv6 L3VPN (with SRv6 End.DT46) -> kernel >= 5.14 -> config example here

[liuyuefengcn]

@ale0524 ok, thanks.