search

FRRouting / frr

The FRRouting Protocol Suite
https://frrouting.org/
Other
3.31k stars 1.25k forks source link

Routes not accepted when adjusting BGP local-preference with inbound route-map #10700

Closed johnfzc closed 2 years ago

johnfzc commented 2 years ago

FRR Local Preference

System Details:

I am trying to use a route-map to control local preferences on a eBGP connection. I created a simple route map with a single entry that sets the preference, when this is applied I stop accepting the route from the peer. I believe that I'm doing something basic very wrong, but I can't understand what.

Here's the critical elements of the most basic route-map I created:

router bgp 65000
 neighbor 169.254.66.33 remote-as 55271
 address-family ipv4 unicast
  neighbor 169.254.66.33 route-map set-loc-pref-170 in

route-map set-loc-pref-170 permit 10
 set local-preference 170

When I apply the route-map inbound on my peer no routes are accepted. Looking at the route-map counters it appears that we do not invoke the entry that should set the local preference:

ELS002T8# sh route-map set-loc-pref-170
ZEBRA:
route-map: set-loc-pref-170 Invoked: 0 Optimization: enabled Processed Change: false
 permit, sequence 10 Invoked 0
  Match clauses:
  Set clauses:
  Call clause:
  Action:
    Exit routemap
BGP:
route-map: set-loc-pref-170 Invoked: 2 Optimization: enabled Processed Change: false
 permit, sequence 10 Invoked 0
  Match clauses:
  Set clauses:
    local-preference 170
  Call clause:
  Action:
    Exit routemap

And we do not accept the route from the peer:

ELS002T8# sh ip bgp summ
[...]
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
169.254.66.1    4      55271      1572      1443        0    0    0 00:01:20            1        1
169.254.66.17   4      55271      1549      1407        0    0    0 00:01:20            1        1
169.254.66.33   4      55271      1772      1628        0    0    0 00:01:20            0        1
169.254.66.49   4      55271      1372      1249        0    0    0 00:01:19            1        1

Removing the route-map for this peer we see the route received:

ELS002T8# sh ip bgp summ
[...]
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
169.254.66.1    4      55271      1552      1424        0    0    0 00:00:07            1        1
169.254.66.17   4      55271      1529      1388        0    0    0 00:00:07            1        1
169.254.66.33   4      55271      1752      1608        0    0    0 00:00:06            1        1
169.254.66.49   4      55271      1352      1230        0    0    0 00:00:06            1        1

I then tried to use a match statement using a prefix-list and with an ACL, the preference modification was not applied to the peer.

route-map set-loc-pref-190 permit 10
 match ip address permit-all
 set local-preference 190
!
route-map set-loc-pref-190 permit 20

I can get the route to appear if I include a second empty permit entry in the route-map, as I did with set-loc-pref-190. Again we don't see the first entry in the route-map being invoked:

Here are the counters on the access-list, you'll notice that the first entry at sequence 10 is not invoked:

ELS002T8# sh route-map set-loc-pref-190
ZEBRA:
route-map: set-loc-pref-190 Invoked: 0 Optimization: enabled Processed Change: false
 permit, sequence 10 Invoked 0
  Match clauses:
    ip address permit-all
  Set clauses:
  Call clause:
  Action:
    Exit routemap
 permit, sequence 20 Invoked 0
  Match clauses:
  Set clauses:
  Call clause:
  Action:
    Exit routemap
BGP:
route-map: set-loc-pref-190 Invoked: 1 Optimization: enabled Processed Change: false
 permit, sequence 10 Invoked 0
  Match clauses:
    ip address permit-all
  Set clauses:
    local-preference 190
  Call clause:
  Action:
    Exit routemap
 permit, sequence 20 Invoked 1
  Match clauses:
  Set clauses:
  Call clause:
  Action:
    Exit routemap

As I can't use the local-preference attribute I end up with ECMP across all available paths, however I do have a preference on paths that I want to use:

B>* 169.254.50.0/24 [20/0] via 169.254.66.1, xfrm810, weight 1, 00:01:29
  *                      via 169.254.66.17, xfrm811, weight 1, 00:01:29
  *                      via 169.254.66.49, xfrm813, weight 1, 00:01:29

Here's a more complete view of the configuration.

router bgp 65000
 bgp router-id 169.254.55.5
 no bgp ebgp-requires-policy
 neighbor 169.254.66.1 remote-as 55271
 neighbor 169.254.66.1 timers connect 30
 neighbor 169.254.66.17 remote-as 55271
 neighbor 169.254.66.17 timers connect 30
 neighbor 169.254.66.33 remote-as 55271
 neighbor 169.254.66.33 timers connect 30
 neighbor 169.254.66.49 remote-as 55271
 neighbor 169.254.66.49 timers connect 30
 !
 address-family ipv4 unicast
  network 169.254.55.5/32
  neighbor 169.254.66.1 allowas-in
  neighbor 169.254.66.1 prefix-list client-announce out
  neighbor 169.254.66.1 route-map set-loc-pref-190 in
  neighbor 169.254.66.17 allowas-in
  neighbor 169.254.66.17 prefix-list client-announce out
  neighbor 169.254.66.33 allowas-in
  neighbor 169.254.66.33 prefix-list client-announce out
  neighbor 169.254.66.33 route-map set-loc-pref-170 in
  neighbor 169.254.66.49 allowas-in
  neighbor 169.254.66.49 prefix-list client-announce out
 exit-address-family

ip prefix-list permit-all seq 10 permit 0.0.0.0/0 le 32
access-list permit-all seq 5 permit any

route-map set-loc-pref-190 permit 10
 match ip address permit-all
 set local-preference 190
!
route-map set-loc-pref-190 permit 20
!
route-map set-loc-pref-170 permit 10
 set local-preference 170
taspelund commented 2 years ago

Can you get the output of show route-map set-loc-pref-170 prefix?

johnfzc commented 2 years ago

It appears to be defaulting to accepting all v4 and v6 prefixes as I would expect:

ELS002T8# show route-map set-loc-pref-170 prefix
ZEBRA:

IPv4 Prefix                                           Route-map Index List
_______________                                       ____________________
    0.0.0.0/0 (2)
(P)
                                                      set-loc-pref-170 seq 10

IPv6 Prefix                                           Route-map Index List
_______________                                       ____________________
    ::/0 (2)
(P)
                                                      set-loc-pref-170 seq 10

BGP:

IPv4 Prefix                                           Route-map Index List
_______________                                       ____________________
    0.0.0.0/0 (2)
(P)
                                                      set-loc-pref-170 seq 10

IPv6 Prefix                                           Route-map Index List
_______________                                       ____________________
    ::/0 (2)
(P)
                                                      set-loc-pref-170 seq 10
johnfzc commented 2 years ago

I enabled some debugs (route-map, bgp updates and neighbor-events) and see the following, it's pretty clear the route-map is responsible for the issue:

Tue Mar  1 15:41:25 2022 daemon.debug bgpd[22241]: 169.254.66.33 rcvd UPDATE w/ attr: nexthop 169.254.66.33, origin ?, path 55271
Tue Mar  1 15:41:25 2022 daemon.debug bgpd[22241]: 169.254.66.33 rcvd UPDATE wlen 0 attrlen 20 alen 4
Tue Mar  1 15:41:25 2022 daemon.debug bgpd[22241]: Best match route-map: set-loc-pref-170, sequence: 10 for pfx: 169.254.50.0/24, result: no match
Tue Mar  1 15:41:25 2022 daemon.debug bgpd[22241]: Route-map: set-loc-pref-170, prefix: 169.254.50.0/24, result: deny
Tue Mar  1 15:41:25 2022 daemon.debug bgpd[22241]: 169.254.66.33 rcvd UPDATE about 169.254.50.0/24 IPv4 unicast -- DENIED due to: route-map;
Tue Mar  1 15:41:25 2022 daemon.debug bgpd[22241]: 169.254.66.33 rcvd UPDATE wlen 0 attrlen 0 alen 0

I still don't understand why this is happening, the configuration I'm using seems very basic. Thanks very much for any help you can offer me.

johnfzc commented 2 years ago

I tried switching to use prepending instead of local-preference:

route-map prepend-2 permit 10 
 set as-path prepend 55271 55271

router bgp 65000
 address-family ipv4 unicast
  neighbor 169.254.66.33 route-map prepend-2 in

And see the same behaviour:

ELS002T8# sh ip bgp summ
[...]
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
169.254.66.1    4      55271      1354      1230        0    0    0 20:25:45            1        1
169.254.66.17   4      55271      1357      1229        0    0    0 20:25:45            1        1
169.254.66.33   4      55271      1368      1248        0    0    0 00:00:15            0        1
169.254.66.49   4      55271      1354      1229        0    0    0 20:25:44            1        1

Total number of neighbors 4
johnfzc commented 2 years ago

I spun two new test x86 based OpenWrt routers running 21.02.2 with FRR 7.5-4 (version based on OpenWrt package version) and was able to make local preference function properly:

wrt-bgp-1# sh ip bgp
BGP table version is 5, local router ID is 192.168.100.134, vrf id 0
Default local pref 100, local AS 65000
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 169.254.1.1/32   0.0.0.0                  0         32768 i
*> 169.254.1.2/32   192.168.100.135          0    190      0 65001 i

I'm planning to try to upgrade FRR on my embedded devices to see if it resolves the issue.

I used the following configuration:

router bgp 65000
 no bgp ebgp-requires-policy
 neighbor 192.168.100.135 remote-as 65001
 !
 address-family ipv4 unicast
  network 169.254.1.1/32
  neighbor 192.168.100.135 route-map set-loc-pref-190 in
 exit-address-family
!
route-map set-loc-pref-190 permit 10
 set local-preference 190
ton31337 commented 2 years ago

I've just checked with the master version and 7.5.1, I can't reproduce with the same configs you gave. Do you reproduce this in a non-OpenWRT environment?

ton31337 commented 2 years ago

@frrbot autoclose in 1 week.

frrbot[bot] commented 2 years ago

This issue will be automatically closed in the specified period unless there is further activity.

johnfzc commented 2 years ago

Thanks for looking at this and attempting to reproduce it @ton31337 , I'm still investigating on my side; sorry that I didn't update the issue.

Currently I'm trying to build an updated version of FRR for my current platform to confirm that it resolves the issue.

frrbot[bot] commented 2 years ago

This issue will no longer be automatically closed.

johnfzc commented 2 years ago

In my original testing I was using FRR 7.5.1 as packaged in OpenWrt commit https://github.com/openwrt/packages/commit/2e4c264 .

This is on a Big Endian mips platform with the following details:

root@ELS002T8:/tmp/frr# uname -a
Linux ELS002T8 5.4.142 #0 Wed Aug 25 08:24:33 2021 mips GNU/Linux
root@ELS002T8:/tmp/frr# cat /proc/cpuinfo
system type             : Qualcomm Atheros QCA9533 ver 2 rev 0
machine                 : GL.iNet GL-X300B
processor               : 0
cpu model               : MIPS 24Kc V7.4
BogoMIPS                : 432.53
wait instruction        : yes
microsecond timers      : yes
tlb_entries             : 16
extra interrupt vector  : yes
hardware watchpoint     : yes, count: 4, address/irw mask: [0x0ffc, 0x0ffc, 0x0ffb, 0x0ffb]
isa                     : mips1 mips2 mips32r1 mips32r2
ASEs implemented        : mips16
Options implemented     : tlb 4kex 4k_cache prefetch mcheck ejtag llsc dc_aliases perf_cntr_intr_bit perf
shadow register sets    : 1
kscratch registers      : 0
package                 : 0
core                    : 0
VCED exceptions         : not available
VCEI exceptions         : not available

My successful test was on x86 using FRR 7.5 as packaged in OpenWrt Commit https://github.com/openwrt/packages/commit/adda7b8 .

I then built the same version of 7.5 on the mips platform and still see the same issue.

I am currently trying to build 8.1.0 as packaged in OpenWrt commit https://github.com/openwrt/packages/commit/7f5dc09 .

I have previously seen issues with other networking packages handling big endian systems like this mips board poorly.

I'll update shortly with the results of this test; please tell me if you think I should explore other avenues.

johnfzc commented 2 years ago

I've updated to 8.1 as described above, the behaviour I'm seeing is much the same, I'm seeing the following debug messages when processing the route-map.

Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [Z2KFR-ZGJC8] 169.254.55.33 rcvd UPDATE w/ attr: nexthop 169.254.55.33, origin ?, path 55271
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [P9VY3-5B6NX] 169.254.55.33 rcvd UPDATE wlen 0 attrlen 20 alen 4
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [MT1SJ-WEJQ1] Best match route-map: prepend-2, sequence: 10 for pfx: 169.254.50.0/24, result: no match
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [H5AW4-JFYQC] Route-map: prepend-2, prefix: 169.254.50.0/24, result: deny
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [G759B-M5T1Y] 169.254.55.33 rcvd UPDATE about 169.254.50.0/24 IPv4 unicast -- DENIED due to: route-map;
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [P9VY3-5B6NX] 169.254.55.33 rcvd UPDATE wlen 0 attrlen 0 alen 0
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [G3V0W-FSWRC] bgp_best_path_select_defer: processing route for IPv4 Unicast : cnt 0
Fri Mar  4 15:21:13 2022 daemon.info bgpd[4913]: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv4 Unicast from 169.254.55.33 in vrf default
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [WJC21-WPGZF] u1:s5 announcing routes to 169.254.55.33, combined into 1 peers
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [VWD21-4A1ZJ] 169.254.10.1 [Update:SEND] 169.254.50.0/24 is filtered
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [TN0HX-6G1RR] u1:s5 send UPDATE w/ attr: nexthop 0.0.0.0, origin i, mp_nexthop ::, metric 0, path
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [HVRWP-5R9NQ] u1:s5 send UPDATE 10.171.254.16/32 IPv4 unicast
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [WEV7K-2GAQ5] u1:s5 send UPDATE len 56 (max message len: 4096) numpfx 1
Fri Mar  4 15:21:13 2022 daemon.debug bgpd[4913]: [MBFVT-8GSC6] u1:s5 169.254.55.33 send UPDATE w/ nexthop 169.254.10.34

I'm not really sure how to proceed from here; cross compiling directly from source outside of the openwrt tree would seem like a good option but I do not know how to do this. I still wonder if the issue is endian related.

I could also attempt to deploy the configuration on a freshly installed router running the latest snapshot, in case some other older library is the cause of my problem.

Please let me know if you can suggest something I can try to address this issue. Thanks very much!

ton31337 commented 2 years ago

Please show the full config.

johnfzc commented 2 years ago

Here's the full configuration:

frr version 8.1
frr defaults traditional
hostname ELS002T8
log stdout
log syslog
!
debug route-map
!
password zebra
!
interface br-lan
 no link-detect
exit

router bgp 65000
 bgp router-id 169.254.55.5
 no bgp ebgp-requires-policy
 neighbor 169.254.66.1 remote-as 55271
 neighbor 169.254.66.1 timers connect 30
 neighbor 169.254.66.17 remote-as 55271
 neighbor 169.254.66.17 timers connect 30
 neighbor 169.254.66.33 remote-as 55271
 neighbor 169.254.66.33 timers connect 30
 neighbor 169.254.66.49 remote-as 55271
 neighbor 169.254.66.49 timers connect 30
 !
 address-family ipv4 unicast
  network 169.254.55.5/32
  neighbor 169.254.66.1 allowas-in
  neighbor 169.254.66.1 prefix-list client-announce out
  neighbor 169.254.66.1 route-map set-loc-pref-190 in
  neighbor 169.254.66.17 allowas-in
  neighbor 169.254.66.17 prefix-list client-announce out
  neighbor 169.254.66.33 allowas-in
  neighbor 169.254.66.33 prefix-list client-announce out
  neighbor 169.254.66.33 route-map prepend-2 in
  neighbor 169.254.66.49 allowas-in
  neighbor 169.254.66.49 prefix-list client-announce out
 exit-address-family
exit
!
access-list vty seq 5 permit 127.0.0.0/8
access-list vty seq 10 deny any
access-list permit-all seq 5 permit any
!
ip prefix-list client-announce seq 10 permit 1.1.1.1/32
ip prefix-list client-announce seq 20 permit 169.254.55.5/32
ip prefix-list permit-all seq 10 permit 0.0.0.0/0 le 32
!
route-map set-loc-pref-190 permit 10
 match ip address permit-all
 set local-preference 190
exit
!
route-map set-loc-pref-190 permit 20
exit
!
route-map set-loc-pref-180 permit 10
 set local-preference 180
exit
!
route-map set-loc-pref-170 permit 10
 set local-preference 170
exit
!
route-map set-loc-pref-160 permit 10
 set local-preference 160
exit
!
route-map prepend-2 permit 10
 set as-path prepend 55271 55271
exit
!
line vty
 access-class vty
exit
!
end
ton31337 commented 2 years ago

Can't replicate on Raspberry as well, but seems this is really related to MIPS. Are you able to provide a testing box (OpenWRT) with SSH access to debugging?

johnfzc commented 2 years ago

Direct remote access by SSH will be challenging but not impossible to provide; screen sharing to the reproduced issue is trivial however. I will contact you on slack to see what you need and try to provide it.

I have been working on trying to reproduce the issue using qemu-system-mips.

ton31337 commented 2 years ago

Reproducible on qemu?

johnfzc commented 2 years ago

I can't yet confirm, installing qemu mips is proving challenging. If it is I will be able to provide you with a self contained test system and we'll be able to confirm that the issue isn't specific to some aspect of OpenWrt.

I've tried to message you on slack, screen sharing would be far easier if it's possible, I could provide you with access now.

johnfzc commented 2 years ago

I was able to recreate the issue using OpenWrt running on qemu mips, here's the BGP adjacency before applying the route-map:

wrt-bgp-1(config-route-map)# do sh ip bgp summ
[...]
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
192.168.1.11    4      65001         5         5        0    0    0 00:00:12            1        2 N/A

And after:

wrt-bgp-1(config-router-af)# do sh ip bgp summ
[...]
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
192.168.1.11    4      65001         9        10        0    0    0 00:00:37            0        1 N/A

You can download drive images and the kernel here, compressed with bzip2:

kernel wrt-bgp-1 wrt-bgp-2

I ran them with the following commands, I wasn't able to make qemu-mips work inside Virtual Machine Manager:

qemu-system-mips -M malta -hda wrt-mips-bgp-1.img -kernel openwrt-malta-be-vmlinux.elf -nographic -append root=/dev/sda console=ttyS0 -net nic,macaddr=00:16:3e:00:00:01 -net tap
qemu-system-mips -M malta -hda wrt-mips-bgp-2.img -kernel openwrt-malta-be-vmlinux.elf -nographic -append root=/dev/sda console=ttyS0 -net nic,macaddr=00:16:3e:00:00:02 -net tap

You will need to confirm the IP address allocated to your machines either by looking at your DHCP server or by pressing enter a few times to get a console and then running ip a s. Both systems are configured with no firewall on their WAN interface and can be accessed using ssh as root without a password. Do not expose them to the Internet ;)

You will need to update the IP addresses in your BGP configuration in order to properly reflect the IPs allocated to your two routers. They should then announce two loopback addresses to each other in the 169.254.1 address space. Adding and removing the route-map should permit or deny this.

I wasn't successful in getting debian to run on mips but may be able to try again later today as it would help to confirm that the issue isn't specific to OpenWrt.

johnfzc commented 2 years ago

I've built a debian mips system and I'm unable to recreate the issue using the OS packaged frr 6.0.2.

I'm attempting to update to a current version of frr to see if the issue can be reproduced.

johnfzc commented 2 years ago

I built FRR 7.5 on an emulated mips debian buster system from the git branch debian/7.5 and was able to recreate the issue:

frr-mips(config-router)# do sh ip bgp summ
[...]
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
192.168.1.10    4      65000        11        13        0    0    0 00:00:08            1        1

Total number of neighbors 1
frr-mips(config-router)# neighbor 192.168.1.10 route-map set-loc-pref-190 in
frr-mips(config-router)# do clear ip bgp * soft
frr-mips(config-router)# do sh ip bgp summ
[...]
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
192.168.1.10    4      65000        13        16        0    0    0 00:00:31            0        0

This serves as confirmation that the issue:

I expect that a full Debian system will be a much more comfortable environment for diagnosing the issue than OpenWrt.

You can download the bzip2 compressed components of the qemu image for this instance here:

kernel initrd disk

I ran the VM using the following command:

sudo qemu-system-mips -kernel vmlinux-4.19.0-18-4kc-malta -initrd initrd.img-4.19.0-18-4kc-malta -hda debian-frr.img -append "root=/dev/sda1 console=ttyS0" -nographic -M malta -m 512 -net tap -net nic,macaddr=00:16:3e:00:00:03

There is a user named router with sudo privs as well as the root user, both have a password of frr. Build artifacts and the source for the compilation of frr and libyang can be found in /home/router/src/.

Under heavy network load the emulated MIPS system behaves pretty poorly, and shows very high pings (>1s) and slow transfer. I used qemu-nbd to mount the disk and perform the git checkout. Heavy CPU usage, like building frr, is shockingly slow, even on a decent (Ryzen 3700 X, entry NVMe disk) system.

Please let me know what else I can provide to assist with diagnosing this issue. Thanks very much for any help you can offer.

ton31337 commented 2 years ago

Thanks, I'll try later.

ton31337 commented 2 years ago

I tested with a big-endian system:

[root@cypris]: ~ $ lscpu                        
Architecture:          ppc
  CPU op-mode(s):      32-bit
  Byte Order:          Big Endian

Can't see this issue with the master version...

Will try testing with your images.

samburney commented 2 years ago

Is this potentially related to the issue fixed in #9782?

johnfzc commented 2 years ago

Reading PR https://github.com/FRRouting/frr/pull/10730 it seems quite likely, I will try testing an updated version; I believe that this patch is part of 8.4 but it's hard for me to tell. If it isn't please let me know.

Thanks!

ton31337 commented 2 years ago

This fix is included in 8.3.1 as well, could you test it?

johnfzc commented 2 years ago

I'm sorry that I haven't had the chance to test it yet, I will do so this week. Thanks for the confirmation that it's included in 8.3.1 @ton31337.

johnfzc commented 2 years ago

Building and installing frr 8.3.1 took longer than I expected, sorry about that. I've completed my testing and confirm that this issue is no longer present in 8.3.1. Before we close this issue can you please confirm which frr releases contain this fix? I've reviewed the release notes but I don't see how the fix is represented. Is the fix in 8.2.2 or only in the 8.3 releases?

ton31337 commented 2 years ago

Building and installing frr 8.3.1 took longer than I expected, sorry about that. I've completed my testing and confirm that this issue is no longer present in 8.3.1. Before we close this issue can you please confirm which frr releases contain this fix? I've reviewed the release notes but I don't see how the fix is represented. Is the fix in 8.2.2 or only in the 8.3 releases?

Only >= 8.3.1.

johnfzc commented 2 years ago

Great, thank you very much for the rapid confirmation, appreciate all of your help with this issue @ton31337 and @samburney.