search

FRRouting / frr

The FRRouting Protocol Suite
https://frrouting.org/
Other
3.21k stars 1.24k forks source link

NHRP redirect does not work with configured cisco NHRP Authentication #16371

Closed aapostoliuk closed 1 month ago

aapostoliuk commented 2 months ago

Description

If cisco nhrp authentication is configured and FRR is a DMVPN HUB, NHRP redirect does not work. Without cisco nhrp authentication everything works as expected.

Version

vyos# show ver
FRRouting 10.2-dev (vyos) on Linux(6.6.36-amd64-vyos).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--sbindir=/usr/lib/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--enable-rpki' '--enable-scripting' '--enable-pim6d' '--disable-grpc' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'

How to reproduce

Network topology. two Spokes - Cisco Routers SPOKE-101 NBMA IP - 192.168.100.11/24 TUNNEL IP - 10.0.0.11/24 Local Network - 10.0.101.0/24

SPOKE-103 NBMA IP - 192.168.100.13/24 TUNNEL IP - 10.0.0.13/24 Local Network - 10.0.103.0/24

HUB - FRR NBMA IP - 192.168.100.100/24 TUNNEL IP - 10.0.0.1/32

All routers are in one network. NBMA Network 192.168.100.0/24

OSPF is running Netfilter is used on FRR side.

Configurations: HUB configuration:

ip addr

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 0c:d5:dc:f4:00:00 brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    altname ens4
    inet 192.168.100.100/24 brd 192.168.100.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::ed5:dcff:fef4:0/64 scope link
       valid_lft forever preferred_lft forever
10: tun100@eth0: <UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
    inet 10.0.0.1/32 scope global tun100
       valid_lft forever preferred_lft forever
    inet6 fe80::f062:5fff:fec1:e293/64 scope link
       valid_lft forever preferred_lft forever

nft

table ip dmvpn_multicast {
        chain OUTPUT {
                type filter hook output priority filter; policy accept;
                oifname "tun100" ip daddr 224.0.0.0/24 counter packets 399 bytes 28968 log group 2
                oifname "tun100" ip daddr 224.0.0.0/24 counter packets 399 bytes 28968 drop
        }
}
table ip dmvpn_hub {
        chain FORWARD {
                type filter hook forward priority filter; policy accept;
                iifname "tun100" oifname "tun100" meter loglimit-0 size 65535 { ip daddr & 255.255.255.0 . ip saddr & 255.255.255.0 timeout 1m limit rate 4/minute burst 1 packets } counter packets 4 bytes 336 log group 1
        }
}

FRR config

vyos# show run
Building configuration...

Current configuration:
!
frr version 10.2-dev
frr defaults traditional
hostname vyos
log syslog
log facility local7
nhrp nflog-group 1
nhrp multicast-nflog-group 2
service integrated-vtysh-config
!
debug nhrp common
debug nhrp interface
debug nhrp kernel
debug nhrp route
debug nhrp event
!
interface tun100
 ip address 10.0.0.1/32
 ip nhrp authentication test123
 ip nhrp holdtime 450
 ip nhrp map multicast dynamic
 ip nhrp network-id 1
 ip nhrp redirect
 ip nhrp registration no-unique
 ip ospf dead-interval 40
 ip ospf network point-to-multipoint
 no link-detect
 tunnel source eth0
exit
!
router ospf
 ospf router-id 1.1.1.1
 auto-cost reference-bandwidth 100
 timers throttle spf 200 1000 10000
 network 10.0.0.0/24 area 0
exit
!
rpki
exit
!
end

SPOKE-101 configuration


interface Tunnel100
 ip address 10.0.0.11 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication test123
 ip nhrp network-id 1
 ip nhrp holdtime 450
 ip nhrp nhs 10.0.0.1 nbma 192.168.100.100 multicast
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 ip ospf network point-to-multipoint
 ip ospf dead-interval 40
 ip ospf hello-interval 10
 tunnel source GigabitEthernet0/0
 tunnel mode gre multipoint
 tunnel key 42
!
interface GigabitEthernet0/0
 ip address 192.168.100.111 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
!
interface GigabitEthernet0/1
 ip address 10.0.101.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
!
router ospf 1
 router-id 1.1.1.2
 passive-interface GigabitEthernet0/1
 network 10.0.0.0 0.0.0.255 area 0
 network 10.0.101.0 0.0.0.255 area 0

SPOKE-103 configuration:

interface Tunnel100
 ip address 10.0.0.13 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp authentication test123
 ip nhrp network-id 1
 ip nhrp holdtime 450
 ip nhrp nhs 10.0.0.1 nbma 192.168.100.100 multicast
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 ip ospf network point-to-multipoint
 ip ospf dead-interval 40
 ip ospf hello-interval 10
 tunnel source GigabitEthernet0/0
 tunnel mode gre multipoint
 tunnel key 42
!
interface GigabitEthernet0/0
 ip address 192.168.100.13 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
!
interface GigabitEthernet0/1
 ip address 10.0.103.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45

!
router ospf 1
 router-id 1.1.1.13
 network 10.0.0.0 0.0.0.255 area 0
 network 10.0.103.0 0.0.0.255 area 0

Trying to ping from host behind SPOKE-103 to host behind SPOKE-101 (from 10.0.103.2 to 10.0.101.2) Debug from FRR HUB:

2024-07-12 09:23:24.967 [DEBG] nhrpd: [W3FAQ-B51V0] Netlink-log: Received msg_type 1024, msg_flags 0
2024-07-12 09:23:24.967 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.103.2/32: zebra route dev tun100
2024-07-12 09:23:24.967 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-12 09:23:24.967 [DEBG] nhrpd: [XRSX6-2JRY1] Send Traffic Indication to 192.168.100.13 (online=1) about packet to 10.0.103.2
2024-07-12 09:23:24.967 [DEBG] nhrpd: [PTQ80-8JY6C] Send Traffic-Indication(8) 10.0.0.1 -> 10.0.103.2
2024-07-12 09:23:24.967 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.13
2024-07-12 09:23:24.967 [DEBG] nhrpd: [W3FAQ-B51V0] Netlink-log: Received msg_type 1024, msg_flags 0
2024-07-12 09:23:24.967 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.101.2/32: zebra route dev tun100
2024-07-12 09:23:24.967 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.11/32: nhrp_if=tun100
2024-07-12 09:23:24.967 [DEBG] nhrpd: [XRSX6-2JRY1] Send Traffic Indication to 192.168.100.111 (online=1) about packet to 10.0.101.2
2024-07-12 09:23:24.967 [DEBG] nhrpd: [PTQ80-8JY6C] Send Traffic-Indication(8) 10.0.0.1 -> 10.0.101.2
2024-07-12 09:23:24.967 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.111
2024-07-12 09:23:24.967 [DEBG] nhrpd: [W3FAQ-B51V0] Netlink-log: Received msg_type 3, msg_flags 0
2024-07-12 09:23:24.988 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.111 -> 192.168.100.100
2024-07-12 09:23:24.988 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-12 09:23:24.988 [DEBG] nhrpd: [RHB3H-QNGNH] Processing Authentication Extension for (test123:test123|0)
2024-07-12 09:23:24.988 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.103.2/32: zebra route dev tun100
2024-07-12 09:23:24.988 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-12 09:23:24.988 [DEBG] nhrpd: [PTQ80-8JY6C] FWD-FAIL Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-12 09:23:24.988 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.13 -> 192.168.100.100
2024-07-12 09:23:24.988 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:23:24.988 [DEBG] nhrpd: [RHB3H-QNGNH] Processing Authentication Extension for (test123:test123|0)
2024-07-12 09:23:24.988 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.101.2/32: zebra route dev tun100
2024-07-12 09:23:24.988 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.11/32: nhrp_if=tun100
2024-07-12 09:23:24.988 [DEBG] nhrpd: [PTQ80-8JY6C] FWD-FAIL Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:23:26.567 [DEBG] nhrpd: [TPNQ6-77EJG] Netlink-mcast-log: Received msg_type 1024, msg_flags 0
2024-07-12 09:23:26.567 [DEBG] nhrpd: [JT71Y-7VYHQ] Intercepted multicast packet leaving tun100 len 72
2024-07-12 09:23:26.567 [DEBG] nhrpd: [PKEHV-MNXHK] Multicast Packet: 192.168.100.100 -> 192.168.100.13, ret = 72, size = 72, addrlen = 4
2024-07-12 09:23:26.567 [DEBG] nhrpd: [PKEHV-MNXHK] Multicast Packet: 192.168.100.100 -> 192.168.100.111, ret = 72, size = 72, addrlen = 4
2024-07-12 09:23:26.652 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.13 -> 192.168.100.100
2024-07-12 09:23:26.652 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:23:26.652 [DEBG] nhrpd: [RHB3H-QNGNH] Processing Authentication Extension for (test123:test123|0)
2024-07-12 09:23:26.652 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.101.2/32: zebra route dev tun100
2024-07-12 09:23:26.652 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.11/32: nhrp_if=tun100
2024-07-12 09:23:26.652 [DEBG] nhrpd: [PTQ80-8JY6C] FWD-FAIL Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:23:26.829 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.111 -> 192.168.100.100
2024-07-12 09:23:26.829 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-12 09:23:26.829 [DEBG] nhrpd: [RHB3H-QNGNH] Processing Authentication Extension for (test123:test123|0)
2024-07-12 09:23:26.829 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.103.2/32: zebra route dev tun100
2024-07-12 09:23:26.829 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-12 09:23:26.829 [DEBG] nhrpd: [PTQ80-8JY6C] FWD-FAIL Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-12 09:23:29.255 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: who-has 10.0.0.13 dev tun100 lladdr 192.168.100.13 nud 0x10 cache used 0 type 4
2024-07-12 09:23:29.255 [DEBG] nhrpd: [QVXNM-NVHEQ] Netlink: update binding for 10.0.0.13 dev tun100 from c (unspec) peer.vc.nbma 192.168.100.13 to lladdr 192.168.100.13
2024-07-12 09:23:29.255 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: new-neigh 10.0.0.13 dev tun100 lladdr 192.168.100.13 nud 0x10 cache used 1 type 4
2024-07-12 09:23:29.255 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: who-has 10.0.0.11 dev tun100 lladdr 192.168.100.111 nud 0x10 cache used 0 type 4
2024-07-12 09:23:29.255 [DEBG] nhrpd: [QVXNM-NVHEQ] Netlink: update binding for 10.0.0.11 dev tun100 from c (unspec) peer.vc.nbma 192.168.100.111 to lladdr 192.168.100.111
2024-07-12 09:23:29.255 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: new-neigh 10.0.0.11 dev tun100 lladdr 192.168.100.111 nud 0x10 cache used 1 type 4
2024-07-12 09:23:29.940 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.13 -> 192.168.100.100
2024-07-12 09:23:29.940 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:23:29.940 [DEBG] nhrpd: [RHB3H-QNGNH] Processing Authentication Extension for (test123:test123|0)
2024-07-12 09:23:29.940 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.101.2/32: zebra route dev tun100
2024-07-12 09:23:29.940 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.11/32: nhrp_if=tun100
2024-07-12 09:23:29.940 [DEBG] nhrpd: [PTQ80-8JY6C] FWD-FAIL Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:23:30.673 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.111 -> 192.168.100.100
2024-07-12 09:23:30.673 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-12 09:23:30.673 [DEBG] nhrpd: [RHB3H-QNGNH] Processing Authentication Extension for (test123:test123|0)
2024-07-12 09:23:30.673 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.103.2/32: zebra route dev tun100
2024-07-12 09:23:30.673 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-12 09:23:30.673 [DEBG] nhrpd: [PTQ80-8JY6C] FWD-FAIL Resolution-Request(1) 10.0.0.11 -> 10.0.103.2

Routing table on spokes

SPOKE-103#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
C        10.0.0.0/24 is directly connected, Tunnel100
O        10.0.0.1/32 [110/1000] via 10.0.0.1, 00:24:18, Tunnel100
O        10.0.0.11/32 [110/1001] via 10.0.0.1, 00:23:57, Tunnel100
L        10.0.0.13/32 is directly connected, Tunnel100
O        10.0.101.0/24 [110/1002] via 10.0.0.1, 00:23:57, Tunnel100
C        10.0.103.0/24 is directly connected, GigabitEthernet0/1
L        10.0.103.1/32 is directly connected, GigabitEthernet0/1
      192.168.100.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.100.0/24 is directly connected, GigabitEthernet0/0
L        192.168.100.13/32 is directly connected, GigabitEthernet0/0

SPOKE-101#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
C        10.0.0.0/24 is directly connected, Tunnel100
O        10.0.0.1/32 [110/1000] via 10.0.0.1, 00:24:53, Tunnel100
L        10.0.0.11/32 is directly connected, Tunnel100
O        10.0.0.13/32 [110/1001] via 10.0.0.1, 00:24:53, Tunnel100
C        10.0.101.0/24 is directly connected, GigabitEthernet0/1
L        10.0.101.1/32 is directly connected, GigabitEthernet0/1
O        10.0.103.0/24 [110/1002] via 10.0.0.1, 00:24:53, Tunnel100
      192.168.100.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.100.0/24 is directly connected, GigabitEthernet0/0
L        192.168.100.111/32 is directly connected, GigabitEthernet0/0

As we can see NHRP redirect is not working. Spokes continue to send traffic through HUB.

PC1> trace 10.0.101.2
trace to 10.0.101.2, 8 hops max, press Ctrl+C to stop
 1   10.0.103.1   1.490 ms  0.926 ms  0.654 ms
 2   10.0.0.1   1.448 ms  1.290 ms  1.336 ms --- HUB
 3   10.0.0.11   3.713 ms  3.458 ms  3.236 ms
 4   *10.0.101.2

Expected behavior

If we do not use the command 'ip nhrp authentication', NHRP redirect works. Debug from FRR:

2024-07-12 09:07:36.551 [DEBG] nhrpd: [TPNQ6-77EJG] Netlink-mcast-log: Received msg_type 1024, msg_flags 0
2024-07-12 09:07:36.551 [DEBG] nhrpd: [JT71Y-7VYHQ] Intercepted multicast packet leaving tun100 len 72
2024-07-12 09:07:36.551 [DEBG] nhrpd: [PKEHV-MNXHK] Multicast Packet: 192.168.100.100 -> 192.168.100.13, ret = 72, size = 72, addrlen = 4
2024-07-12 09:07:36.551 [DEBG] nhrpd: [PKEHV-MNXHK] Multicast Packet: 192.168.100.100 -> 192.168.100.111, ret = 72, size = 72, addrlen = 4
2024-07-12 09:07:36.999 [DEBG] nhrpd: [W3FAQ-B51V0] Netlink-log: Received msg_type 1024, msg_flags 0
2024-07-12 09:07:36.999 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.103.2/32: zebra route dev tun100
2024-07-12 09:07:36.999 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-12 09:07:36.999 [DEBG] nhrpd: [XRSX6-2JRY1] Send Traffic Indication to 192.168.100.13 (online=1) about packet to 10.0.103.2
2024-07-12 09:07:36.999 [DEBG] nhrpd: [PTQ80-8JY6C] Send Traffic-Indication(8) 10.0.0.1 -> 10.0.103.2
2024-07-12 09:07:36.999 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.13
2024-07-12 09:07:36.999 [DEBG] nhrpd: [W3FAQ-B51V0] Netlink-log: Received msg_type 1024, msg_flags 0
2024-07-12 09:07:36.999 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.101.2/32: zebra route dev tun100
2024-07-12 09:07:36.999 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.11/32: nhrp_if=tun100
2024-07-12 09:07:36.999 [DEBG] nhrpd: [XRSX6-2JRY1] Send Traffic Indication to 192.168.100.111 (online=1) about packet to 10.0.101.2
2024-07-12 09:07:36.999 [DEBG] nhrpd: [PTQ80-8JY6C] Send Traffic-Indication(8) 10.0.0.1 -> 10.0.101.2
2024-07-12 09:07:36.999 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.111
2024-07-12 09:07:36.999 [DEBG] nhrpd: [W3FAQ-B51V0] Netlink-log: Received msg_type 3, msg_flags 0
2024-07-12 09:07:37.014 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.13 -> 192.168.100.100
2024-07-12 09:07:37.014 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:07:37.014 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.101.2/32: zebra route dev tun100
2024-07-12 09:07:37.014 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.11/32: nhrp_if=tun100
2024-07-12 09:07:37.014 [DEBG] nhrpd: [M78NA-AFP11] Processing NHRP_EXTENSION_NAT_ADDRESS while forwarding the request packet
2024-07-12 09:07:37.014 [DEBG] nhrpd: [RFX78-JMH2T] Proto is 10.0.0.13
2024-07-12 09:07:37.014 [DEBG] nhrpd: [MFKFP-TR5FR] c->cur.remote_nbma_natoa is (unspec)
2024-07-12 09:07:37.014 [DEBG] nhrpd: [PTQ80-8JY6C] Send Resolution-Request(1) 10.0.0.13 -> 10.0.101.2
2024-07-12 09:07:37.014 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.111
2024-07-12 09:07:37.014 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.111 -> 192.168.100.100
2024-07-12 09:07:37.014 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-12 09:07:37.014 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.103.2/32: zebra route dev tun100
2024-07-12 09:07:37.014 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-12 09:07:37.014 [DEBG] nhrpd: [M78NA-AFP11] Processing NHRP_EXTENSION_NAT_ADDRESS while forwarding the request packet
2024-07-12 09:07:37.014 [DEBG] nhrpd: [RFX78-JMH2T] Proto is 10.0.0.11
2024-07-12 09:07:37.014 [DEBG] nhrpd: [MFKFP-TR5FR] c->cur.remote_nbma_natoa is (unspec)
2024-07-12 09:07:37.014 [DEBG] nhrpd: [PTQ80-8JY6C] Send Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-12 09:07:37.014 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.13
2024-07-12 09:07:37.022 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.13 -> 192.168.100.100
2024-07-12 09:07:37.022 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Reply(2) 10.0.103.2 -> 10.0.0.11
2024-07-12 09:07:37.022 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.11/32: nhrp_if=tun100
2024-07-12 09:07:37.022 [DEBG] nhrpd: [WPDSG-XKZBH] Processing NHRP_EXTENSION_NAT_ADDRESS while forwarding the reply packet
2024-07-12 09:07:37.022 [DEBG] nhrpd: [RFX78-JMH2T] Proto is 10.0.0.13
2024-07-12 09:07:37.022 [DEBG] nhrpd: [MFKFP-TR5FR] c->cur.remote_nbma_natoa is (unspec)
2024-07-12 09:07:37.022 [DEBG] nhrpd: [PTQ80-8JY6C] Send Resolution-Reply(2) 10.0.103.2 -> 10.0.0.11
2024-07-12 09:07:37.022 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.111
2024-07-12 09:07:37.022 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.111 -> 192.168.100.100
2024-07-12 09:07:37.022 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Reply(2) 10.0.101.2 -> 10.0.0.13
2024-07-12 09:07:37.022 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-12 09:07:37.022 [DEBG] nhrpd: [WPDSG-XKZBH] Processing NHRP_EXTENSION_NAT_ADDRESS while forwarding the reply packet
2024-07-12 09:07:37.022 [DEBG] nhrpd: [RFX78-JMH2T] Proto is 10.0.0.11
2024-07-12 09:07:37.022 [DEBG] nhrpd: [MFKFP-TR5FR] c->cur.remote_nbma_natoa is (unspec)
2024-07-12 09:07:37.022 [DEBG] nhrpd: [PTQ80-8JY6C] Send Resolution-Reply(2) 10.0.101.2 -> 10.0.0.13
2024-07-12 09:07:37.022 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.13

SPOKES Routing table

SPOKE-103#show ip route next-hop-override
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
C        10.0.0.0/24 is directly connected, Tunnel100
O        10.0.0.1/32 [110/1000] via 10.0.0.1, 00:01:23, Tunnel100
O   %    10.0.0.11/32 [110/1001] via 10.0.0.1, 00:01:23, Tunnel100
                      [NHO][110/255] via 10.0.0.11, 00:01:12, Tunnel100
L        10.0.0.13/32 is directly connected, Tunnel100
O   %    10.0.101.0/24 [110/1002] via 10.0.0.1, 00:01:23, Tunnel100
                       [NHO][110/255] via 10.0.0.11, 00:01:12, Tunnel100
C        10.0.103.0/24 is directly connected, GigabitEthernet0/1
L        10.0.103.1/32 is directly connected, GigabitEthernet0/1
      192.168.100.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.100.0/24 is directly connected, GigabitEthernet0/0
L        192.168.100.13/32 is directly connected, GigabitEthernet0/0

SPOKE-101#show ip route next-hop-override
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
C        10.0.0.0/24 is directly connected, Tunnel100
O        10.0.0.1/32 [110/1000] via 10.0.0.1, 00:02:46, Tunnel100
L        10.0.0.11/32 is directly connected, Tunnel100
O   %    10.0.0.13/32 [110/1001] via 10.0.0.1, 00:02:36, Tunnel100
                      [NHO][110/255] via 10.0.0.13, 00:02:25, Tunnel100
C        10.0.101.0/24 is directly connected, GigabitEthernet0/1
L        10.0.101.1/32 is directly connected, GigabitEthernet0/1
O   %    10.0.103.0/24 [110/1002] via 10.0.0.1, 00:02:36, Tunnel100
                       [NHO][110/255] via 10.0.0.13, 00:02:25, Tunnel100
      192.168.100.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.100.0/24 is directly connected, GigabitEthernet0/0
L        192.168.100.111/32 is directly connected, GigabitEthernet0/0

Trace

PC1> trace 10.0.101.2
trace to 10.0.101.2, 8 hops max, press Ctrl+C to stop
 1   10.0.103.1   1.089 ms  0.919 ms  0.947 ms
 2   10.0.0.11   3.331 ms  2.720 ms  2.311 ms
 3   *10.0.101.2   1.588 ms

As we can see NHRP redirect works as expected and Spokes send traffic directly to each other without using HUB.

Actual behavior

Results are in "How to reproduce" section.

Additional context

No response

Checklist

fett0 commented 1 month ago

guys well , this is last bug that we found ,when FRR is working as a DMVPN HUB, NHRP redirect does not work. not register communication spoke to spoke (without authentication ip nhrp authentication) works as expected, Could you take at look ? @dleroy @volodymyrhuti thanks you!

dleroy commented 1 month ago

guys well , this is last bug that we found ,when FRR is working as a DMVPN HUB, NHRP redirect does not work. not register communication spoke to spoke (without authentication ip nhrp authentication) works as expected, Could you take at look ? @dleroy @volodymyrhuti thanks you!

I'm actively looking into this one.

aapostoliuk commented 1 month ago

@dleroy @Jafaral @volodymyrhuti I have tested this fix but it does not solve this issue here is my debug from FRR as a HUB

2024-07-22 09:21:05.096 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.111 -> 192.168.100.100
2024-07-22 09:21:05.096 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-22 09:21:05.096 [DEBG] nhrpd: [RHB3H-QNGNH] Processing Authentication Extension for (test123:test123|0)
2024-07-22 09:21:05.096 [DEBG] nhrpd: [KNPB6-NP2Y4] lookup 10.0.103.2/32: zebra route dev tun100
2024-07-22 09:21:05.096 [DEBG] nhrpd: [GVZF0-990Z5] lookup 10.0.0.13/32: nhrp_if=tun100
2024-07-22 09:21:05.096 [DEBG] nhrpd: [M78NA-AFP11] Processing NHRP_EXTENSION_NAT_ADDRESS while forwarding the request packet
2024-07-22 09:21:05.096 [DEBG] nhrpd: [RFX78-JMH2T] Proto is 10.0.0.11
2024-07-22 09:21:05.096 [DEBG] nhrpd: [MFKFP-TR5FR] c->cur.remote_nbma_natoa is (unspec)
2024-07-22 09:21:05.096 [DEBG] nhrpd: [PTQ80-8JY6C] Send Resolution-Request(1) 10.0.0.11 -> 10.0.103.2
2024-07-22 09:21:05.096 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.13
2024-07-22 09:21:05.103 [DEBG] nhrpd: [K0534-5VD2M] PACKET: Recv 192.168.100.13 -> 192.168.100.100
2024-07-22 09:21:05.103 [DEBG] nhrpd: [PTQ80-8JY6C] Recv Error-Indication(7) 10.0.0.13 -> 10.0.0.11
2024-07-22 09:21:05.103 [DEBG] nhrpd: [PTQ80-8JY6C] Send Error-Indication(7) 10.0.0.11 -> 10.0.0.13
2024-07-22 09:21:05.103 [DEBG] nhrpd: [WSA6E-5GM0H] PACKET: Send 192.168.100.100 -> 192.168.100.13
2024-07-22 09:21:05.103 [INFO] nhrpd: [PRQ0A-R3YY1] From 192.168.100.13: error: authentication failure
2024-07-22 09:21:06.549 [DEBG] nhrpd: [TPNQ6-77EJG] Netlink-mcast-log: Received msg_type 1024, msg_flags 0
2024-07-22 09:21:06.549 [DEBG] nhrpd: [JT71Y-7VYHQ] Intercepted multicast packet leaving tun100 len 72
2024-07-22 09:21:06.549 [DEBG] nhrpd: [PKEHV-MNXHK] Multicast Packet: 192.168.100.100 -> 192.168.100.13, ret = 72, size = 72, addrlen = 4
2024-07-22 09:21:06.549 [DEBG] nhrpd: [PKEHV-MNXHK] Multicast Packet: 192.168.100.100 -> 192.168.100.111, ret = 72, size = 72, addrlen = 4
2024-07-22 09:21:07.317 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: who-has 10.0.0.13 dev tun100 lladdr 192.168.100.13 nud 0x10 cache used 0 type 4
2024-07-22 09:21:07.317 [DEBG] nhrpd: [QVXNM-NVHEQ] Netlink: update binding for 10.0.0.13 dev tun100 from c (unspec) peer.vc.nbma 192.168.100.13 to lladdr 192.168.100.13
2024-07-22 09:21:07.317 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: new-neigh 10.0.0.13 dev tun100 lladdr 192.168.100.13 nud 0x10 cache used 1 type 4
2024-07-22 09:21:07.317 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: who-has 10.0.0.11 dev tun100 lladdr 192.168.100.111 nud 0x10 cache used 0 type 4
2024-07-22 09:21:07.317 [DEBG] nhrpd: [QVXNM-NVHEQ] Netlink: update binding for 10.0.0.11 dev tun100 from c (unspec) peer.vc.nbma 192.168.100.111 to lladdr 192.168.100.111
2024-07-22 09:21:07.317 [DEBG] nhrpd: [QQ0NK-1H449] Netlink: new-neigh 10.0.0.11 dev tun100 lladdr 192.168.100.111 nud 0x10 cache used 1 type 4
aapostoliuk commented 1 month ago

Here is debug from Cisco SPOKE

*Jul 22 09:36:18.893: NHRP-ATTR: ext_type: 32775, ext_len : 11
*Jul 22 09:36:18.893: NHRP-ATTR: ext_type: 32768, ext_len : 0
*Jul 22 09:36:18.894: NHRP: Receive Traffic Indication via Tunnel100 vrf global(0x0), packet size: 143
*Jul 22 09:36:18.894:  (F) afn: AF_IP(1), type: IP(800), hop: 1, ver: 1
*Jul 22 09:36:18.895:      shtl: 4(NSAP), sstl: 0(NSAP)
*Jul 22 09:36:18.895:      pktsz: 143 extoff: 124
*Jul 22 09:36:18.895:  (M) traffic code: redirect(0)
*Jul 22 09:36:18.895:      src NBMA: 192.168.100.100
*Jul 22 09:36:18.895:      src protocol: 10.0.0.1, dst protocol: 10.0.101.2
*Jul 22 09:36:18.896:      Contents of nhrp traffic indication packet:
*Jul 22 09:36:18.896:         45 00 00 54 28 15 00 00 3E 01 74 90 0A 00 65 02
*Jul 22 09:36:18.896:         0A 00 67 02 00 00 12 E3 15 28 00 01 08 09 0A 0B
*Jul 22 09:36:18.896:         0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B
*Jul 22 09:36:18.896:         1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B
*Jul 22 09:36:18.897:         2C 2D 2E 2F 30 31 32 33 34 35 36 37 38 39 3A 3B
*Jul 22 09:36:18.897:         3C 3D 3E
*Jul 22 09:36:18.897: Authentication Extension(7):
*Jul 22 09:36:18.897:   type:Cleartext(1), data:test123
*Jul 22 09:36:18.897: NHRP-DETAIL: netid_in = 1, to_us = 0
*Jul 22 09:36:18.898: NHRP: nhrp_rtlookup yielded GigabitEthernet0/1
SPOKE-101#
*Jul 22 09:36:18.898: NHRP-DETAIL: netid_out 0, netid_in 1
*Jul 22 09:36:18.898: NHRP: Parsing NHRP Traffic Indication
*Jul 22 09:36:18.899: NHRP: Enqueued NHRP Resolution Request for destination: 10.0.103.2
*Jul 22 09:36:18.899: NHRP: Checking for delayed event NULL/10.0.103.2 on list (Tunnel100 vrf: global(0x0))
*Jul 22 09:36:18.899: NHRP: No delayed event node found.
SPOKE-101#
*Jul 22 09:36:22.772: NHRP: Checking for delayed event NULL/10.0.103.2 on list (Tunnel100 vrf: global(0x0))
*Jul 22 09:36:22.772: NHRP: No delayed event node found.
*Jul 22 09:36:22.772: NHRP: There is no VPE Extension to construct for the request
*Jul 22 09:36:22.773: NHRP: Sending NHRP Resolution Request for dest: 10.0.103.2 to nexthop: 10.0.103.2 using our src: 10.0.0.11 vrf:global(0x0)
*Jul 22 09:36:22.773: NHRP: Attempting to send packet through interface Tunnel100 via DEST  dst 10.0.103.2
*Jul 22 09:36:22.774: NHRP: Send Resolution Request via Tunnel100 vrf global(0x0), packet size: 87
*Jul 22 09:36:22.774:  src: 10.0.0.11, dst: 10.0.103.2
*Jul 22 09:36:22.775:  (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Jul 22 09:36:22.775:      shtl: 4(NSAP), sstl: 0(NSAP)
*Jul 22 09:36:22.775:      pktsz: 87 extoff: 52
*Jul 22 09:36:22.775:  (M) flags: "router auth src-stable nat ", reqid: 7
*Jul 22 09:36:22.775:      src NBMA: 192.168.100.111
*Jul 22 09:36:22.775:      src protocol: 10.0.0.11, dst protocol: 10.0.103.2
*Jul 22 09:36:22.776:  (C-1) code: no error(0)
*Jul 22 09:36:22.776:        prefix: 32, mtu: 17912, hd_time: 450
*Jul 22 09:36:22.776:        addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 255
*Jul 22 09:36:22.776: Responder Address Extension(3):
SPOKE-101#
*Jul 22 09:36:22.777: Forward Transit NHS Record Extension(4):
*Jul 22 09:36:22.777: Reverse Transit NHS Record Extension(5):
*Jul 22 09:36:22.777: Authentication Extension(7):
*Jul 22 09:36:22.777:   type:Cleartext(1), data:test123
*Jul 22 09:36:22.777: NAT address Extension(9):
*Jul 22 09:36:22.777: NHRP-DETAIL: Unable to get dst from pak sb
*Jul 22 09:36:22.778: NHRP: Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 192.168.100.100
*Jul 22 09:36:22.778: NHRP: 115 bytes out Tunnel100
*Jul 22 09:36:22.957: NHRP-ATTR: ext_type: 32771, ext_len : 0
*Jul 22 09:36:22.957: NHRP-ATTR: ext_type: 32772, ext_len : 20
*Jul 22 09:36:22.957: NHRP-ATTR: ext_type: 32773, ext_len : 0
*Jul 22 09:36:22.958: NHRP-ATTR: ext_type: 32775, ext_len : 0
*Jul 22 09:36:22.958: NHRP-ERROR: Incorrect Auth extn length 0
*Jul 22 09:36:22.959: NHRP: Sending error indication. Reason: 'Pak sanity failure' LINE: 9877
*Jul 22 09:36:22.959: NHRP: Attempting to send packet through interface Tunnel100 via DEST  dst 10.0.0.13
*Jul 22 09:36:22.960: NHRP: Send Error Indication via Tunnel100 vrf global(0x0), packet size: 151
*Jul 22 09:36:22.960:  src: 10.0.0.11, dst: 10.0.0.13
*Jul 22 09:36:22.961:  (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1
*Jul 22 09:36:22.961:      shtl: 4(NSAP), sstl: 0(NSAP)
*Jul 22 09:36:22.961:      pktsz: 151 extoff: 0
*Jul 22 09:36:22.961:  (M) error code: protocol generic error(7), offset: 84
*Jul 22 09:36:22.961:      src NBMA: 192.168.100.111
*Jul 22 09:36:22.962:      src protocol: 10.0.0.11, dst protocol: 10.0.0.13
*Jul 22 09:36:22.962:      Contents of error packet:
*Jul 22 09:36:22.962:         00 01 08 00 00 00 00 00 00 FE 00 6F 39 07 00 34
*Jul 22 09:36:22.963:         01 01 04 00 04 04 C8 02 00 00 00 0E C0 A8 64 0D
*Jul 22 09:36:22.963:         0A 00 00 0D 0A 00 65 02
*Jul 22 09:36:22.963:
*Jul 22 09:36:22.963:
*Jul 22 09:36:22.963: NHRP-DETAIL: Unable to get dst from pak sb
*Jul 22
SPOKE-101#09:36:22.963: NHRP: Encapsulation succeeded.  Sending NHRP Control Packet  NBMA Address: 192.168.100.100
*Jul 22 09:36:22.964: NHRP: 179 bytes out Tunnel100
*Jul 22 09:36:22.977: NHRP-ERROR: Packet Recved with 0 Hop counts on Tunnel100.
aapostoliuk commented 1 month ago

NHRP_RESOLUTION_REQUEST2.dmp I added the tcpdump file with NHRP packets. I see there 2 fields of "NHRP Authentication Extension" in the NHRP Resolution Request. One of them has "Extension length=0".