Closed vincentmli closed 11 months ago
by the way, I am testing suricata AF_XDP
maybe suricata AF_XDP is not setup to use xdp dispatcher ?
src/source-af-xdp.c
/* Socket configuration */
ptv->xsk.cfg.rx_size = XSK_RING_CONS__DEFAULT_NUM_DESCS;
ptv->xsk.cfg.tx_size = XSK_RING_PROD__DEFAULT_NUM_DESCS;
ptv->xsk.cfg.xdp_flags = afxdpconfig->mode;
ptv->xsk.cfg.bind_flags = afxdpconfig->bind_flags;
Yeah, libxdp multiprog only works if every XDP user on the system uses it (or a compatible library). The AF_XDP support in libxdp itself does use the dispatcher, so any application that has upgraded from the older libbpf support to libxdp will automatically co-exist with other programs through the dispatcher.
If Suricata does not use libxdp there is not much that can be done from the libxdp side; maybe submit a patch to Suricata to use libxdp for their AF_XDP attachment?
Thanks @tohojo for replying :)
I had a look at Suricata code source-af-xdp.c L430-L446 , it calls xsk_socket__create
which should eventually calls __xsk_setup_xdp_prog
that should use dispatcher.
I also looked at bpf-examples xdpsock.c L1017-L1030 which also uses xsk_socket__create
, and it uses dispatcher, so I am not sure what config setting is required to allow xsk_socket__create
to use dispatcher, no need to set
libxdp_flags
or libbpf_flags
, right? maybe something else in Suricata stops it using dispatcher, still looking...
it seems my libbpf version (0.7) mixed up with libxdp installation, and suricata might be confused too, after I re-install libbpf latest version, re-install libxdp, and re-install suricata, the issue is gone
/usr/bin/suricata -c /etc/suricata/suricata.yaml --af-xdp=eno2 -vvv
Notice: suricata: This is Suricata version 7.0.1-dev (4044d0287 2023-08-31) running in SYSTEM mode [LogVersion:suricata.c:1154]
Perf: af-xdp: 2 cores, so using 2 threads [ConfigSetThreads:runmode-af-xdp.c:125]
Info: runmodes: eno2: creating 1 thread [RunModeSetLiveCaptureWorkersForDevice:util-runmodes.c:255]
libbpf: elf: skipping unrecognized data section(8) .xdp_run_config
libbpf: elf: skipping unrecognized data section(9) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
Config: flow-manager: using 1 flow manager threads [FlowManagerThreadSpawn:flow-manager.c:948]
Config: flow-manager: using 1 flow recycler threads [FlowRecyclerThreadSpawn:flow-manager.c:1154]
Info: unix-manager: unix socket '/var/run/suricata/suricata-command.socket' [UnixNew:unix-manager.c:136]
Notice: threads: Threads created -> W: 1 FM: 1 FR: 1 Engine started. [TmThreadWaitOnThreadRunning:tm-threads.c:1890]
xdp-loader status shows xdp dispatcher being used
xdp-loader status
CURRENT XDP PROGRAM STATUS:
Interface Prio Program name Mode ID Tag Chain actions
--------------------------------------------------------------------------------------
lo <No XDP program loaded!>
enp1s0f0 <No XDP program loaded!>
enp1s0f1 <No XDP program loaded!>
eno1 <No XDP program loaded!>
eno2 xdp_dispatcher skb 532 90f686eb86991928
=> 20 xsk_def_prog 541 8f9c40757cb0a6a2 XDP_PASS
br0 <No XDP program loaded!>
wg0 <No XDP program loaded!>
virbr0 <No XDP program loaded!>
I use xdp-loader to load XDP program on an interface that is already attached with AF_XDP, got error below:
xdp-loader load eno2 -m skb ./xdp_pass.o libxdp: Generating multi-prog dispatcher for 1 programs libxdp: Existing program is not using a dispatcher, can't replace; unload first Couldn't attach XDP program on iface 'eno2': Device or resource busy(-16)
the program id
38
is used by AF_XDPuse case is like packet -> XDP DDOS -> AF_XDP, is that possible?