search

xdp-project / bpf-examples

Making eBPF programming easier via build env and examples
437 stars 85 forks source link

xdp-synproxy vlan support ? #107

Open boythx opened 1 year ago

vincentmli commented 1 year ago

you mean allow the program to parse vlan header? could you describe more detail about the vlan use case?

boythx commented 1 year ago

Thankyou for reply

I get worng about vlan.

My system network

root@xdp2:~# ip route show table firewall
default nhid 6 proto static metric 20
    nexthop via 10.9.8.20 dev bond0.908 weight 1
    nexthop via 10.9.8.21 dev bond0.908 weight 1
10.9.8.16/29 dev bond0.908 proto kernel scope link src 10.9.8.22
local 10.9.8.22 dev bond0.908 proto kernel scope host src 10.9.8.22
broadcast 10.9.8.23 dev bond0.908 proto kernel scope link src 10.9.8.22
13.9.11.4 dev bond0.544 scope link
root@xdp2:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: enp3s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 40:a6:b7:34:aa:98 brd ff:ff:ff:ff:ff:ff
10: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 40:a6:b7:34:aa:98 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::42a6:b7ff:fe34:aa98/64 scope link
       valid_lft forever preferred_lft forever
14: bond0.544@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master firewall state UP group default qlen 1000
    link/ether 40:a6:b7:34:aa:98 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::42a6:b7ff:fe34:aa98/64 scope link
       valid_lft forever preferred_lft forever
17: bond0.908@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpgeneric/id:5324 qdisc noqueue master firewall state UP group default qlen 1000
    link/ether 40:a6:b7:34:aa:98 brd ff:ff:ff:ff:ff:ff
    inet 10.9.8.22/29 scope global bond0.908
       valid_lft forever preferred_lft forever
    inet6 fe80::42a6:b7ff:fe34:aa98/64 scope link
       valid_lft forever preferred_lft forever

tcpdump on machine firewall befor load xdp synproxy

root@xdp2:~# tcpdump -eni any net 13.9.11.4 and port 80
...
13:49:27.166333 enp3s0f0 In  ifindex 2 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 84: 184.22.105.45.5187 > 13.9.11.4.80: Flags [SEW], seq 887826975, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 2746211936 ecr 0,sackOK,eol], length 0
13:49:27.166334 bond0 In  ifindex 10 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 84: 184.22.105.45.5187 > 13.9.11.4.80: Flags [SEW], seq 887826975, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 2746211936 ecr 0,sackOK,eol], length 0
13:49:27.166335 bond0.908 In  ifindex 17 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 84: 184.22.105.45.5187 > 13.9.11.4.80: Flags [SEW], seq 887826975, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 2746211936 ecr 0,sackOK,eol], length 0
13:49:27.166350 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 84: 184.22.105.45.5187 > 13.9.11.4.80: Flags [SEW], seq 887826975, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 2746211936 ecr 0,sackOK,eol], length 0
13:49:27.166352 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 84: 184.22.105.45.5187 > 13.9.11.4.80: Flags [SEW], seq 887826975, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 2746211936 ecr 0,sackOK,eol], length 0
13:49:27.166354 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 84: 184.22.105.45.5187 > 13.9.11.4.80: Flags [SEW], seq 887826975, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 2746211936 ecr 0,sackOK,eol], length 0
13:49:27.176732 enp3s0f0 In  ifindex 2 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3967980254, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 0
13:49:27.176732 bond0 In  ifindex 10 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 0
13:49:27.176733 bond0.908 In  ifindex 17 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 0
13:49:27.176739 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 0
13:49:27.176739 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 0
13:49:27.176741 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 0
13:49:27.176747 enp3s0f0 In  ifindex 2 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 162: 184.22.105.45.5187 > 13.9.11.4.80: Flags [P.], seq 0:90, ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 90
13:49:27.176747 bond0 In  ifindex 10 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 162: 184.22.105.45.5187 > 13.9.11.4.80: Flags [P.], seq 0:90, ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 90
13:49:27.176748 bond0.908 In  ifindex 17 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 162: 184.22.105.45.5187 > 13.9.11.4.80: Flags [P.], seq 0:90, ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 90
13:49:27.176752 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 162: 184.22.105.45.5187 > 13.9.11.4.80: Flags [P.], seq 0:90, ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 90
13:49:27.176752 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 162: 184.22.105.45.5187 > 13.9.11.4.80: Flags [P.], seq 0:90, ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 90
13:49:27.176753 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 162: 184.22.105.45.5187 > 13.9.11.4.80: Flags [P.], seq 0:90, ack 1, win 2056, options [nop,nop,TS val 2746211953 ecr 1164345302], length 90
13:49:27.186736 enp3s0f0 In  ifindex 2 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2003, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186737 bond0 In  ifindex 10 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2003, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186738 bond0.908 In  ifindex 17 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2003, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186739 enp3s0f0 In  ifindex 2 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186740 bond0 In  ifindex 10 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186740 bond0.908 In  ifindex 17 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186754 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2003, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186755 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2003, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186757 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2003, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186761 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186762 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.186763 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.187269 enp3s0f0 In  ifindex 2 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [F.], seq 90, ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.187270 bond0 In  ifindex 10 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [F.], seq 90, ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.187270 bond0.908 In  ifindex 17 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [F.], seq 90, ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.187279 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [F.], seq 90, ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.187280 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [F.], seq 90, ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.187282 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [F.], seq 90, ack 3349, win 2048, options [nop,nop,TS val 2746211963 ecr 1164345369], length 0
13:49:27.195766 enp3s0f0 In  ifindex 2 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3350, win 2048, options [nop,nop,TS val 2746211974 ecr 1164345379], length 0
13:49:27.195767 bond0 In  ifindex 10 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3350, win 2048, options [nop,nop,TS val 2746211974 ecr 1164345379], length 0
13:49:27.195767 bond0.908 In  ifindex 17 0c:29:ef:d4:2d:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3350, win 2048, options [nop,nop,TS val 2746211974 ecr 1164345379], length 0
13:49:27.195776 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3350, win 2048, options [nop,nop,TS val 2746211974 ecr 1164345379], length 0
13:49:27.195777 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3350, win 2048, options [nop,nop,TS val 2746211974 ecr 1164345379], length 0
13:49:27.195778 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.5187 > 13.9.11.4.80: Flags [.], ack 3350, win 2048, options [nop,nop,TS val 2746211974 ecr 1164345379], length 0

What i do...

1. when load xdp to interface enp3s0f0 XDP_TX not working

root@client1:~# curl 13.9.11.4:80
curl: (28) Failed to connect to 13.9.11.4 port 80 after 75012 ms: Couldn't connect to server

tcpdump on machine firewall after load xdp synproxy to interface enp3s0f0

root@xdp2:~# tcpdump -eni any net 13.9.11.4 and port 80
    tcpdump: data link type LINUX_SLL2
    tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
    listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
    ^C
    0 packets captured
    1 packet received by filter
    0 packets dropped by kernel

2. when load xdp to interface bond0.908 XDP_TX working but show xdpgeneric/id:5324

root@client1:~# curl 13.9.11.4:80
curl: (56) Recv failure: Connection reset by peer

tcpdump on machine firewall after load xdp synproxy to interface bond0.908

root@xdp2:~# tcpdump -eni any net 13.9.11.4 and port 80
...
15:35:05.704801 enp3s0f0 In  ifindex 2 0c:29:ef:d4:66:22 ethertype IPv4 (0x0800), length 84: 184.22.105.45.60905 > 13.9.11.4.80: Flags [SEW], seq 1670546161, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 208776965 ecr 0,sackOK,eol], length 0
15:35:05.704803 bond0 In  ifindex 10 0c:29:ef:d4:66:22 ethertype IPv4 (0x0800), length 84: 184.22.105.45.60905 > 13.9.11.4.80: Flags [SEW], seq 1670546161, win 65535, options [mss 1412,nop,wscale 6,nop,nop,TS val 208776965 ecr 0,sackOK,eol], length 0
15:35:05.704835 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 80: 13.9.11.4.80 > 184.22.105.45.60905: Flags [S.E], seq 1831558014, ack 1670546162, win 0, options [mss 1460,sackOK,TS val 92549558 ecr 208776965,nop,wscale 7], length 0
15:35:05.704839 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 80: 13.9.11.4.80 > 184.22.105.45.60905: Flags [S.E], seq 1831558014, ack 1670546162, win 0, options [mss 1460,sackOK,TS val 92549558 ecr 208776965,nop,wscale 7], length 0
15:35:05.716285 enp3s0f0 In  ifindex 2 0c:29:ef:d4:66:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.60905 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 208776976 ecr 92549558], length 0
15:35:05.716286 bond0 In  ifindex 10 0c:29:ef:d4:66:22 ethertype IPv4 (0x0800), length 72: 184.22.105.45.60905 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 208776976 ecr 92549558], length 0
15:35:05.716294 bond0.908 In  ifindex 17 0c:29:ef:d4:66:22 ethertype IPv4 (0x0800), length 100: 184.22.105.45.60905 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 208776976 ecr 92549558], length 0
15:35:05.716314 bond0.544 Out ifindex 14 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.60905 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 208776976 ecr 92549558], length 0
15:35:05.716316 bond0 Out ifindex 10 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.60905 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 208776976 ecr 92549558], length 0
15:35:05.716318 enp3s0f0 Out ifindex 2 40:a6:b7:34:aa:98 ethertype IPv4 (0x0800), length 72: 184.22.105.45.60905 > 13.9.11.4.80: Flags [.], ack 1, win 2056, options [nop,nop,TS val 208776976 ecr 92549558], length 0

But real system i needed interface use enp3s0f0

Plase help me Thank you

vincentmli commented 1 year ago

What i do...

1. when load xdp to interface enp3s0f0 XDP_TX not working

root@client1:~# curl 13.9.11.4:80
curl: (28) Failed to connect to 13.9.11.4 port 80 after 75012 ms: Couldn't connect to server

so you have route in firewall table

13.9.11.4 dev bond0.544 scope link

just want to be clear, is 13.9.11.4 a remote server that is routed/firewalled through firewall machine bond0.544 vlan interface? and you want xdp synproxy on enp3s0f0 ? if so, do you have proper iptables rules setup for SYNPROXY, and DNAT...etc? what is the exact steps you deployed xdp synproxy?

tcpdump on machine firewall after load xdp synproxy to interface enp3s0f0

root@xdp2:~# tcpdump -eni any net 13.9.11.4 and port 80
    tcpdump: data link type LINUX_SLL2
    tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
    listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
    ^C
    0 packets captured
    1 packet received by filter
    0 packets dropped by kernel

it looks the packet is dropped by xdp synproxy which is before tcpdump, so tcpdump will not see the packet dropped by XDP.

2. when load xdp to interface bond0.908 XDP_TX working but show xdpgeneric/id:5324

root@client1:~# curl 13.9.11.4:80
curl: (56) Recv failure: Connection reset by peer

I have not tried XDP on bond interface, not even vlan interface on bond :) to sort out the problem, I suggest to attach to the real network interface first