xdp-project / xdp-tools

Utilities and example programs for use with XDP
Other
653 stars 143 forks source link

creating AF_XDP socket fails on loading single prog. #191

Closed marcos44pl closed 2 years ago

marcos44pl commented 2 years ago

I've encountered few issues when trying to load my own program on interface and attach to the AF_XDP socket using libxdp.

Having simple program:

struct bpf_map_def SEC("maps") xsks_map = {
    .type = BPF_MAP_TYPE_XSKMAP,
    .key_size = sizeof(int),
    .value_size = sizeof(int),
    .max_entries = 1,
};

SEC("xdp_sock") int xdp_sock_prog(struct xdp_md* ctx) {
    return bpf_redirect_map(&xsks_map, ctx->rx_queue_index, 0);
}

char _license[] SEC("license") = "GPL";

Compiled with:

clang
    -I...
    -Wall
    -O2 
    -g 
    -target bpf
    -c xdp_pass_everything_redirect.c
    -o xdp_pass_everything_redirect.o

Then in runtime it succesfully load program and attach it to the proper interface, but it fails on:

_prog = xdp_program__open_file(_name.c_str(), "xdp_sock", NULL);
...
ret = xdp_program__attach(_prog, _if_index, xdp_attach_mode::XDP_MODE_SKB, 0);
...
...
 cfg.libbpf_flags = 0;
 cfg.xdp_flags = XDP_FLAGS_SKB_MODE;
 cfg.bind_flags = XDP_COPY;
 size_t result = xsk_socket__create(&_socket, _if_name.c_str(), 0, _umem.umem_internal, &_rx, &_tx, &cfg);

with errors:

libbpf: elf: skipping unrecognized data section(7) .xdp_run_config
libbpf: elf: skipping unrecognized data section(8) xdp_metadata
libbpf: elf: skipping unrecognized data section(8) xdp_metadata
libbpf: elf: skipping unrecognized data section(8) xdp_metadata
libxdp: Failed to attach program xdp_sock_prog to dispatcher: Operation not supported
libxdp: Falling back to loading single prog without dispatcher
[error]  ERROR creating socket: -17

So it is expected that it doesnt support multi prog facility (kernel <5.10), but I would expect to load that program in single prog mode. I've tried some other approches and when I compiled it without -g, the libxdp couldn't find BTF info and then on trying loading single prog it failed on errc: 2

[error] ERROR creating socket: -2

VERSIONS: Kernel version: 5.9 Clang compiler - 11.0.1 to compile bpf prog GCC compiler - 11.2.0 to compile runtime app Last release libxdp ver: v1.2.3

I also tried the same thing on another machine where I have kernel 5.15 and everything works in multiprog mode, but I still don't understand few things.

Why:

It sounds like problem with backward compatibility to single programs, but maybe I'm doing sth wrong.

tohojo commented 2 years ago

You have to set the XSK_LIBXDP_FLAGS__INHIBIT_PROG_LOAD in the cfg object when creating the socket, or libxdp will try to load the built-in program. With multiprog available that will work (and you'll see both programs) and without multiprog it'll fail because you already attached a program (with EEXIST, which is the error you're getting).

marcos44pl commented 2 years ago

Thank you for the help, I'll check it and let you know

marcos44pl commented 2 years ago

Thanks! I was able to make it working after reviewing xdp examples. For others maybe it is worth to mention that if you use XSK_LIBXDP_FLAGS__INHIBIT_PROG_LOAD flag you need to update xsk_map:

load own program
...
        cfg.libbpf_flags = XSK_LIBXDP_FLAGS__INHIBIT_PROG_LOAD;
...
        size_t result = xsk_socket__create(&_socket, _if_name.c_str(), 0, _umem, &_rx, &_tx, &cfg);
        **auto map = bpf_object__find_map_by_name(bpf_obj, "xsks_map");
    auto xsks_map_fd = bpf_map__fd(map);
        result = xsk_socket__update_xskmap(_socket, xsks_map_fd);**