search

pimvanpelt / lcpng

Linux Control Plane plugin for VPP
Apache License 2.0
85 stars 21 forks source link

Source Based Routing sync abnormal #10

Closed BurlyLuo closed 8 months ago

BurlyLuo commented 8 months ago

With LCP mode, i find an issue that if configured the SBR(source based routing) at kernel side, but if trigger the ping to the dstation ip with source ip, the dpdk-input trace show:

1. When recv the ICMP replay, we can see the ERR:   ip4-local: ip4 source lookup miss

Packet 13

00:05:06:565401: dpdk-input
  fpeth1 rx queue 0
  buffer 0x99395: current data 0, length 98, buffer-pool 0, ref-count 1, trace handle 0xc
                  ext-hdr-valid 
  PKT MBUF: port 0, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0xa4c4e5c0
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0 
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 00:50:56:fd:cf:21 -> 52:54:00:f4:da:da
  ICMP: 114.114.114.114 -> 10.1.5.10
    tos 0x00, ttl 128, length 84, checksum 0x46fd dscp CS0 ecn NON_ECN
    fragment id 0xffbc
  ICMP echo_reply checksum 0x598a id 45648
00:05:06:565410: ethernet-input
  frame: flags 0x1, hw-if-index 1, sw-if-index 1
  IP4: 00:50:56:fd:cf:21 -> 52:54:00:f4:da:da
00:05:06:565414: ip4-input
  ICMP: 114.114.114.114 -> 10.1.5.10
    tos 0x00, ttl 128, length 84, checksum 0x46fd dscp CS0 ecn NON_ECN
    fragment id 0xffbc
  ICMP echo_reply checksum 0x598a id 45648
00:05:06:565415: ip4-lookup
  fib 0 dpo-idx 7 flow hash: 0x00000000
  ICMP: 114.114.114.114 -> 10.1.5.10
    tos 0x00, ttl 128, length 84, checksum 0x46fd dscp CS0 ecn NON_ECN
    fragment id 0xffbc
  ICMP echo_reply checksum 0x598a id 45648
00:05:06:565417: ip4-receive
    ICMP: 114.114.114.114 -> 10.1.5.10
      tos 0x00, ttl 128, length 84, checksum 0x46fd dscp CS0 ecn NON_ECN
      fragment id 0xffbc
    ICMP echo_reply checksum 0x598a id 45648
00:05:06:565418: ip4-drop
    ICMP: 114.114.114.114 -> 10.1.5.10
      tos 0x00, ttl 128, length 84, checksum 0x46fd dscp CS0 ecn NON_ECN
      fragment id 0xffbc
    ICMP echo_reply checksum 0x598a id 45648
00:05:06:565419: error-drop
  rx:fpeth1
00:05:06:565421: drop
  ip4-local: ip4 source lookup miss

  1. kernel side configuration:

    root@localhost:~# ip rule show 
0:      from all lookup local
32765:  from 10.1.5.0/24 lookup 100
32766:  from all lookup main
32767:  from all lookup default
root@localhost:~# ip r s t 100
114.114.114.114 via 10.1.5.254 dev fpeth1 
root@localhost:~#

  2. host environment detailes:

    root@localhost:~# uname -r 
6.1.0-17-amd64
root@localhost:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm
root@localhost:~# vppctl show version 
vpp v24.02-rc0~175-g31d4891cf built by pim on bookworm-builder at 2023-12-09T12:54:52
root@localhost:~#

  3. Extra findings

    If configured with based destation routing, the ping is fine. 
root@localhost:~# ip r a 114.114.114.114/32 via 10.1.5.254 dev fpeth1 
root@localhost:~# ping 114.114.114.114 -I 10.1.5.10
PING 114.114.114.114 (114.114.114.114) from 10.1.5.10 : 56(84) bytes of data.
64 bytes from 114.114.114.114: icmp_seq=1 ttl=128 time=27.2 ms
^C
--- 114.114.114.114 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 27.187/27.187/27.187/0.000 ms
root@localhost:~#

  4. ip fib table and int info

    
root@localhost:~# vppctl show ip fib
ipv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto flowlabel ] epoch:0 flags:none locks:[adjacency:1, recursive-resolution:1, default-route:1, lcp-rt:1, ]
0.0.0.0/0
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:1 buckets:1 uRPF:0 to:[0:0]]
[0] [@0]: dpo-drop ip4
0.0.0.0/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:2 buckets:1 uRPF:1 to:[0:0]]
[0] [@0]: dpo-drop ip4
10.1.5.0/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:10 buckets:1 uRPF:12 to:[0:0]]
[0] [@0]: dpo-drop ip4
10.1.5.0/24
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:9 buckets:1 uRPF:15 to:[0:0]]
[0] [@4]: ipv4-glean: [src:10.1.5.0/24] fpeth1: mtu:9000 next:1 flags:[] ffffffffffff525400f4dada0806
10.1.5.10/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:12 buckets:1 uRPF:16 to:[1170:98280]]
[0] [@13]: dpo-receive: 10.1.5.10 on fpeth1
10.1.5.254/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:22 buckets:1 uRPF:26 to:[0:0]]
[0] [@5]: ipv4 via 10.1.5.254 fpeth1: mtu:9000 next:5 flags:[] 005056fdcf21525400f4dada0800
10.1.5.255/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:11 buckets:1 uRPF:14 to:[13:3126]]
[0] [@0]: dpo-drop ip4
224.0.0.0/4
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:4 buckets:1 uRPF:3 to:[0:0]]
[0] [@0]: dpo-drop ip4
240.0.0.0/4
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:3 buckets:1 uRPF:2 to:[0:0]]
[0] [@0]: dpo-drop ip4
255.255.255.255/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:5 buckets:1 uRPF:11 to:[0:0]]
[0] [@13]: dpo-receive: 0.0.0.0 on local0
ipv4-VRF:100, fib_index:1, flow hash:[src dst sport dport proto flowlabel ] epoch:0 flags:none locks:[CLI:1, lcp-rt:1, ]
0.0.0.0/0
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:16 buckets:1 uRPF:19 to:[0:0]]
[0] [@0]: dpo-drop ip4
0.0.0.0/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:17 buckets:1 uRPF:21 to:[0:0]]
[0] [@0]: dpo-drop ip4
114.114.114.114/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:21 buckets:1 uRPF:28 to:[0:0]]
[0] [@5]: ipv4 via 10.1.5.254 fpeth1: mtu:9000 next:5 flags:[] 005056fdcf21525400f4dada0800
224.0.0.0/4
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:19 buckets:1 uRPF:23 to:[0:0]]
[0] [@0]: dpo-drop ip4
240.0.0.0/4
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:18 buckets:1 uRPF:22 to:[0:0]]
[0] [@0]: dpo-drop ip4
255.255.255.255/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:20 buckets:1 uRPF:25 to:[0:0]]
[0] [@13]: dpo-receive: 0.0.0.0 on local0
root@localhost:~#

root@localhost:~# vppctl show int Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
fpeth1 1 up 9000/0/0/0 rx packets 1356 rx bytes 145446 tx packets 1245 tx bytes 118892 drops 1345 ip4 1294 ip6 3 fpeth2 2 down 9000/0/0/0
local0 0 down 0/0/0/0
tap0 3 up 9000/0/0/0 rx packets 1239 rx bytes 118468 tx packets 49 tx bytes 3126 drops 2 ip4 1176 ip6 20 root@localhost:~#

vpp# show lcp
lcp default netns lcp lcp-auto-subint on lcp lcp-sync on itf-pair: [0] fpeth1 tap0 fpeth1 7 type tap

pimvanpelt commented 8 months ago

This is an unsupported configuration.

Linux CP does not configure source based routing, the equivalent of which is called ACL Based Forwarding in VPP: https://s3-docs.fd.io/vpp/23.06/cli-reference/clis/clicmd_src_plugins_abf.html?highlight=abf

Please reach out to the vpp developer mailinglist to discuss further, and take note of the warning in the README of this repo.

BurlyLuo commented 8 months ago

Sure, thanks a lot.