search

FRRouting / frr

The FRRouting Protocol Suite
https://frrouting.org/
Other
3.31k stars 1.25k forks source link

BGP Route Advertise(update/withdraw) is happening immediately and not waiting for MRAI for the peer #11337

Closed arunsesh closed 2 years ago

arunsesh commented 2 years ago

To Reproduce router bgp 5555 bgp router-id 20.20.20.20 no bgp ebgp-requires-policy bgp graceful-restart-disable no bgp network import-check neighbor 10.40.66.1 remote-as 5000 neighbor 10.40.66.1 update-source 10.40.10.91 neighbor 10.40.66.1 advertisement-interval 90 <<<< address-family ipv4 unicast network 14.14.14.14/32 network 13.13.13.13/32 <<<<<<<<<< add/remove this network exit-address-family !

Updates (Advertise/Withdraw) of routes happening immediately to the peer.

swdev-hng012-red1# show ip bgp neighbors 10.40.66.1 BGP neighbor is 10.40.66.1, remote AS 5000, local AS 6000 no-prepend replace-as, external link Hostname: hng-aseshadri BGP version 4, remote router ID 10.60.54.166, local router ID 20.20.20.20 BGP state = Established, up for 00:01:01 Last read 00:00:01, Last write 00:00:01 Hold time is 180, keepalive interval is 60 seconds Neighbor capabilities: 4 Byte AS: advertised and received Extended Message: advertised AddPath: IPv4 Unicast: RX advertised IPv4 Unicast and received Route refresh: advertised and received(old & new) Enhanced Route Refresh: advertised Address Family IPv4 Unicast: advertised and received Hostname Capability: advertised (name: swdev-hng012-red1,domain name: n/a) received (name: hng-aseshadri,domain name: n/a) Graceful Restart Capability: received Remote Restart timer is 120 seconds Address families by peer: none Graceful restart information: End-of-RIB send: IPv4 Unicast End-of-RIB received: Local GR Mode: Disable* Remote GR Mode: NotApplicable R bit: False Timers: Configured Restart Time(sec): 120 Received Restart Time(sec): 120 Message statistics: Inq depth is 0 Outq depth is 0 Sent Rcvd Opens: 740 413 Notifications: 617 670 Updates: 52 7 Keepalives: 1888 1855 Route Refresh: 0 3 Capability: 0 0 Total: 3297 2948 Minimum time between advertisement runs is 90 seconds <<<<< Update source is 10.40.10.91

For address family: IPv4 Unicast Update group 31, subgroup 19 Packet Queue length 0 Community attribute sent to this neighbor(all) 0 accepted prefixes

Connections established 19; dropped 18 Last reset 00:01:03, User reset Message received that caused BGP to send a NOTIFICATION: FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF 00560104 138800B4 0A3C36A6 39020601 04000100 01020280 00020202 00020641 04000013 88020645 04000101 01021149 0F0D686E 672D6173 65736861 64726900 02044002 8078 Local host: 10.40.10.91, Local port: 43750 Foreign host: 10.40.66.1, Foreign port: 179 Nexthop: 10.40.10.91 Nexthop global: fe80::5054:ff:fe7c:2b80 Nexthop local: fe80::5054:ff:fe7c:2b80 BGP connection: shared network BGP Connect Retry Timer in Seconds: 120 Estimated round trip time: 4 ms Read thread: on Write thread: on FD used: 31

ton31337 commented 2 years ago

This is already fixed in master, please verify.

arunsesh commented 2 years ago

Thanks @ton31337. Let me build from master and check.

arunsesh commented 2 years ago

Not sure if the build generated @ https://ci1.netdef.org/artifact/FRR-FRR82REL/shared/build-5/CentOS-7-x86_64-RPMs/frr-8.2.2-02.el7.x86_64.rpm is good enough. Checked and found updates/withdraw go immediately. Had MRAI configured to 15 secs

ton31337 commented 2 years ago

I misunderstood your issue. Actually, FRR sends initial updates bypassing the MRAI timer, this is the actual implementation. If you need to delay initial updates as well, you have to use http://docs.frrouting.org/en/stable-8.2/bgp.html?highlight=update-delay#clicmd-bgp-update-delay-MAX-DELAY.

arunsesh commented 2 years ago

1> The 13.13.13.13/32 was added after sometime of BGP peering. Are you saying this will be sent immediately and not wait until MRAI ?

2> The 13.13.13.13/32 was removed after it was advertised. And update happened immediately. For this the fix was given (https://github.com/FRRouting/frr/commit/b10b6d527200c92bef9d576c8c90eaf7098aed06). This seems to be broken now.

ton31337 commented 2 years ago

  1. No. If this is an initial start of the BGPd, then MRAI is skipped and updates are forced to be sent.

  2. If we remove 13.13.13.13/32 after the peering is up and running, and add 13.13.13.13/32 again. Then updates are sent only after the MRAI timer expires. This is how it works in master at the moment (verified). Or are you concerned about immediate withdrawals?

arunsesh commented 2 years ago

Yes. 13.13.13.13/32 added/removed after peering is up is sent immediately and not waiting until MRAI expiry. Tested on https://ci1.netdef.org/artifact/FRR-FRR82REL/shared/build-5/CentOS-7-x86_64-RPMs/frr-8.2.2-02.el7.x86_64.rpm as i was facing some setup issues to compile from master.

ton31337 commented 2 years ago

Are you about immediate withdrawals or updates? If you remove the route, it's removed immediately (not waiting for MRAI). If you re-add it, MRAI started and the update is pending until MRAI expires.

arunsesh commented 2 years ago

This is already fixed in master, please verify. There was a confusion on how this MRAI works.

Tested on latest master - https://ci1.netdef.org/browse/FRR-FRR-5757/artifact/shared/CentOS-7-x86_64-RPMs/ too. And it seems to working.

Here is the understanding, After sending an update, the MRAI timer ticks down. And if any updates /withdraw rcvd, it will be sent after MRAI timer expiry.

hng-aseshadri(config-router)# network 15.14.14.107/32 hng-aseshadri(config-router)# do sh ip bgp neighbors 10.40.10.91 json | include mrai "mraiInterval":90, "mraiTimerExpireInMsecs":71000, hng-aseshadri(config-router)# no network 15.14.14.106/32 hng-aseshadri(config-router)# do sh ip bgp neighbors 10.40.10.91 json | include mrai "mraiInterval":90, "mraiTimerExpireInMsecs":23000, hng-aseshadri(config-router)# do sh ip bgp neighbors 10.40.10.91 json | include mrai

`2022/06/03 18:08:18 BGP: [HVRWP-5R9NQ] u2:s2 send UPDATE 15.14.14.106/32 IPv4 unicast 2022/06/03 18:08:18 BGP: [WEV7K-2GAQ5] u2:s2 send UPDATE len 57 (max message len: 65535) numpfx 1 2022/06/03 18:08:18 BGP: [MBFVT-8GSC6] u2:s2 10.40.10.91 send UPDATE w/ nexthop 10.40.66.1 2022/06/03 18:08:20 BGP: [P8XN0-33WQ6] 10.40.10.91 [FSM] Timer (keepalive timer expire) 2022/06/03 18:08:20 BGP: [HRDT0-0DPQ7] 10.40.10.91 sending KEEPALIVE 2022/06/03 18:08:20 BGP: [X61A3-E95TJ] 10.40.10.91 KEEPALIVE rcvd 2022/06/03 18:08:29 BGP: [K423X-ETGCQ] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=15.14.14.107/32 2022/06/03 18:08:29 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=15.14.14.107/32, selected=0x202e120 2022/06/03 18:08:43 BGP: [K423X-ETGCQ] group_announce_route_walkcb: afi=IPv4, safi=unicast, p=15.14.14.106/32 2022/06/03 18:08:43 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=15.14.14.106/32, selected=0x0

2022/06/03 18:09:20 BGP: [P8XN0-33WQ6] 10.40.10.91 [FSM] Timer (keepalive timer expire) 2022/06/03 18:09:20 BGP: [HRDT0-0DPQ7] 10.40.10.91 sending KEEPALIVE 2022/06/03 18:09:20 BGP: [X61A3-E95TJ] 10.40.10.91 KEEPALIVE rcvd 2022/06/03 18:09:48 BGP: [P3D3N-3277A] 10.40.10.91 [FSM] Timer (routeadv timer expire) 2022/06/03 18:09:48 BGP: [WHJ94-YXNE1] u2:s2 send UPDATE 15.14.14.106/32 IPv4 unicast -- unreachable 2022/06/03 18:09:48 BGP: [G259R-NCBEJ] u2:s2 send UPDATE (withdraw) len 28 numpfx 1 2022/06/03 18:09:48 BGP: [TN0HX-6G1RR] u2:s2 send UPDATE w/ attr: nexthop 0.0.0.0, origin i, mp_nexthop ::, localpref 100, metric 0, path 2022/06/03 18:09:48 BGP: [HVRWP-5R9NQ] u2:s2 send UPDATE 15.14.14.107/32 IPv4 unicast 2022/06/03 18:09:48 BGP: [WEV7K-2GAQ5] u2:s2 send UPDATE len 57 (max message len: 65535) numpfx 1 2022/06/03 18:09:48 BGP: [MBFVT-8GSC6] u2:s2 10.40.10.91 send UPDATE w/ nexthop 10.40.66.1 2022/06/03 18:10:20 BGP: [P8XN0-33WQ6] 10.40.10.91 [FSM] Timer (keepalive timer expire) 2022/06/03 18:10:20 BGP: [HRDT0-0DPQ7] 10.40.10.91 sending KEEPALIVE 2022/06/03 18:10:20 BGP: [X61A3-E95TJ] 10.40.10.91 KEEPALIVE rcvd 2022/06/03 18:11:20 BGP: [P8XN0-33WQ6] 10.40.10.91 [FSM] Timer (keepalive timer expire) 2022/06/03 18:11:20 BGP: [HRDT0-0DPQ7] 10.40.10.91 sending KEEPALIVE 2022/06/03 18:11:20 BGP: [X61A3-E95TJ] 10.40.10.91 KEEPALIVE rcvd `