search

ntop / PF_RING

High-speed packet processing framework
http://www.ntop.org
GNU Lesser General Public License v2.1
2.64k stars 351 forks source link

IP defrag does not works for UDP VLAN traffic #747

Open ates opened 2 years ago

ates commented 2 years ago

Hello,

I've tried to send UDP data which size is more than MTU via VLAN interface and found what IP defragmentation does not works in this case.

Tested on:

options passed to pf_ring.ko: enable_debug=2 enable_ip_defrag=1

[687783.079029] [PF_RING] Welcome to PF_RING 8.1.0 ($Revision: dev:c449372b7f3d6fb4fe2b86c8197814f2e0a14e2c$)
                (C) 2004-20 ntop.org
[687783.079031] LINUX_VERSION_CODE 00050812
[687783.079031] [PF_RING] Min # ring slots 4096
[687783.079032] [PF_RING] Slot version     18
[687783.079032] [PF_RING] Capture TX       Yes [RX+TX]
[687783.079032] [PF_RING] IP Defragment    Yes
[687783.079039] NET: Registered protocol family 27

Logs:

[687863.363816] [PF_RING][DEBUG] ring_create:4515 [pid=805720]
[687863.363820] [PF_RING][DEBUG] ring_insert:2025 ring_insert
[687863.363821] [PF_RING][DEBUG] lockless_list_add:638 BEGIN [total=0]
[687863.363822] [PF_RING][DEBUG] lockless_list_add:664 END [total=1][id=0][top_element_id=0]
[687863.363823] [PF_RING][DEBUG] lockless_list_add:669 [slot 0 is full]
[687863.363827] [PF_RING][DEBUG] ring_proc_add:1196 Added /proc/net/pf_ring/805720-none.1
[687863.363827] [PF_RING][DEBUG] ring_create:4589 created
[687863.363832] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=107)
[687863.363834] [PF_RING][DEBUG] ring_setsockopt:7320 --> SO_RING_BUCKET_LEN=65535
[687863.363835] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=127)
[687863.363847] [PF_RING][DEBUG] ring_bind:5611 ring_bind() called
[687863.363848] [PF_RING][DEBUG] ring_bind:5628 searching device wlp3s0
[687863.363850] [PF_RING][DEBUG] packet_ring_bind:5568 packet_ring_bind(wlp3s0, bucket_len=65535) called
[687863.363851] [PF_RING][DEBUG] ring_proc_remove:1215 Removing /proc/net/pf_ring/805720-none.1
[687863.363852] [PF_RING][DEBUG] ring_proc_remove:1219 Removed /proc/net/pf_ring/805720-none.1
[687863.363854] [PF_RING][DEBUG] ring_proc_add:1196 Added /proc/net/pf_ring/805720-wlp3s0.1
[687863.363855] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=108)
[687863.363856] [PF_RING][DEBUG] ring_setsockopt:7005 Setting channel 0
[687863.363857] [PF_RING][DEBUG] ring_setsockopt:7019 [channel_id_mask=FFFFFFFFFFFFFFFF]
[687863.363858] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=141)
[687863.383754] [PF_RING][DEBUG] ring_mmap:5688 called
[687863.383756] [PF_RING][DEBUG] ring_mmap:5696 called, size: 4096 bytes [bucket_len=65535]
[687863.383757] [PF_RING][DEBUG] ring_alloc_mem:1927 ring_alloc_mem(bucket_len=65535)
[687863.383758] [PF_RING][DEBUG] ring_alloc_mem:1970 [PF_RING] Warning: jumbo mtu or snaplen (65535), resizing slots.. (num_slots = 512 x slot_len = 65592)
[687863.388479] [PF_RING][DEBUG] ring_alloc_mem:1985 successfully allocated 33595392 bytes at 0xffffba434de69000
[687863.388481] [PF_RING][DEBUG] ring_alloc_mem:2002 allocated 512 slots [slot_len=65592][tot_mem=33595392]
[687863.388482] [PF_RING][DEBUG] ring_mmap:5747 mmap [slot_len=65592][tot_slots=512] for ring on device wlp3s0
[687863.388484] [PF_RING][DEBUG] do_memory_mmap:5645 mode=0, size=4096, ptr=00000000e571badb
[687863.388489] [PF_RING][DEBUG] ring_mmap:5812 succeeded
[687863.388511] [PF_RING][DEBUG] ring_mmap:5688 called
[687863.388512] [PF_RING][DEBUG] ring_mmap:5696 called, size: 33595392 bytes [bucket_len=65535]
[687863.388513] [PF_RING][DEBUG] ring_mmap:5747 mmap [slot_len=65592][tot_slots=512] for ring on device wlp3s0
[687863.388513] [PF_RING][DEBUG] do_memory_mmap:5645 mode=0, size=33595392, ptr=00000000e571badb
[687863.389182] [PF_RING][DEBUG] ring_mmap:5812 succeeded
[687863.389188] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=140)
[687863.389190] [PF_RING][DEBUG] is_netdev_promisc:5210 checking promisc for wlp3s0
[687863.389192] [PF_RING][DEBUG] set_netdev_promisc:5224 setting promisc for wlp3s0
[687863.389201] device wlp3s0 entered promiscuous mode
[687863.389374] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(179)
[687863.389376] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(182)
[687863.389381] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(184)
[687863.389386] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=117)
[687863.389387] [PF_RING][DEBUG] ring_setsockopt:7299 --> SO_SET_POLL_WATERMARK=1
[687863.390435] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(184)
[687863.390439] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=26)
[687863.390440] [PF_RING][DEBUG] ring_setsockopt:6897 BPF filter
[687863.390441] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=26)
[687863.390442] [PF_RING][DEBUG] ring_setsockopt:6897 BPF filter
[687863.390475] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=106)
[687863.390476] [PF_RING][DEBUG] ring_setsockopt:7237 * SO_ACTIVATE_RING *
[687900.256909] [PF_RING][DEBUG] add_skb_to_ring:3814 ring_id=1 pfr->filtering_sample_rate=0 pfr->filtering_sampling_size=0
[687900.256946] [PF_RING][DEBUG] add_skb_to_ring:3814 ring_id=1 pfr->filtering_sample_rate=0 pfr->filtering_sampling_size=0
[687903.952629] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=124)
[687903.953426] [PF_RING][DEBUG] ring_release:5362 called ring_release(wlp3s0)
[687903.953428] [PF_RING][DEBUG] ring_proc_remove:1215 Removing /proc/net/pf_ring/805720-wlp3s0.1
[687903.953430] [PF_RING][DEBUG] ring_proc_remove:1219 Removed /proc/net/pf_ring/805720-wlp3s0.1
[687903.953433] [PF_RING][DEBUG] ring_remove:2058 ring_remove()
[687903.953434] [PF_RING][DEBUG] ring_remove:2072 Found socket to remove
[687903.953435] [PF_RING][DEBUG] lockless_list_remove:692 BEGIN [total=1]
[687903.953436] [PF_RING][DEBUG] lockless_list_remove:711 END [total=0][top_element_id=0]
[687903.953437] [PF_RING][DEBUG] ring_remove:2091 leaving ring_remove()
[687903.954067] [PF_RING][DEBUG] unset_netdev_promisc:5247 resetting promisc for wlp3s0
[687903.954073] device wlp3s0 left promiscuous mode
[687904.059835] [PF_RING][DEBUG] ring_release:5513 ring_release: done

How to reproduce:

Interface with VLAN configured on machine 1:

enp0s3.1@enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 08:00:27:db:5d:1f brd ff:ff:ff:ff:ff:ff
    inet 10.4.0.2/8 brd 10.255.255.255 scope global enp0s3.1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fedb:5d1f/64 scope link 
       valid_lft forever preferred_lft forever

Interface with VLAN configured on machine 2:

wlp3s0.1@wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5c:e0:c5:a5:da:66 brd ff:ff:ff:ff:ff:ff
    inet 10.4.0.1/8 brd 10.255.255.255 scope global wlp3s0.1
       valid_lft forever preferred_lft forever
    inet6 fe80::5ee0:c5ff:fea5:da66/64 scope link 
       valid_lft forever preferred_lft forever

Send data from machine 1 to machine 2:

cat /etc/sysctl.conf | nc -u 10.4.0.1 50000

Run tcpdump on machine 2:

tcpdump -i wlp3s0.1 -e -n -vvv
tcpdump: listening on wlp3s0.1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:24:45.441002 08:00:27:db:5d:1f > 5c:e0:c5:a5:da:66, ethertype 802.1Q (0x8100), length 1518: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 32461, offset 0, flags [+], proto UDP (17), length 1500)
    10.4.0.2.41556 > 10.4.0.1.50000: UDP, bad length 2683 > 1472
15:24:45.441036 08:00:27:db:5d:1f > 5c:e0:c5:a5:da:66, ethertype 802.1Q (0x8100), length 1249: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 32461, offset 1480, flags [none], proto UDP (17), length 1231)
    10.4.0.2 > 10.4.0.1: ip-proto-17

Also tested on interfaces without VLAN configured and it's works (sent the same data using nc):

tcpdump -i wlp3s0 -e -n -vvv "udp port 50000"
tcpdump: listening on wlp3s0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:27:18.141203 08:00:27:db:5d:1f > 5c:e0:c5:a5:da:66, ethertype IPv4 (0x0800), length 2725: (tos 0x0, ttl 64, id 22009, offset 0, flags [none], proto UDP (17), length 2711)
    10.10.1.41.54100 > 10.10.1.111.50000: [udp sum ok] UDP, length 2683

Here is pcaps for both cases: pf_ring_defrag_vlan.zip

ates commented 2 years ago

VLAN stripping offload is disabled

rx-vlan-offload: off
tx-vlan-offload: off [requested on]
rx-vlan-filter: on [fixed]