xsk_umem__create() return -22

[LinArcX]

I have this function in my application:

void configUmem()
{
    struct xsk_umem_config config = {
        .fill_size = XSK_RING_PROD__DEFAULT_NUM_DESCS,
        .comp_size = XSK_RING_CONS__DEFAULT_NUM_DESCS,
        .frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE,
    };

    struct xsk_umem *umem = NULL;
    struct xsk_ring_cons cq;
    struct xsk_ring_prod fq;
    void *b;
    int ret;

    if (posix_memalign(&b, getpagesize(), UMEM_SIZE)) {
        perror("posix_memalign failed");
        exit(EXIT_FAILURE);
    }

    if (xsk_umem__create(&umem, b, UMEM_SIZE, &fq, &cq, &config)) {
        perror("xsk_umem__create() failed");
    }
}

xsk_umem__create() always fails with error = -22. why? any idea?

I use: Debian bullseye(Kernel: 6.1.54)

[LinArcX]

maybe it's related to my kernel configurations?

Here are some information:

cat /boot/config-6.1.54 | grep BPF
CONFIG_BPF=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
# BPF subsystem
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT=y
# CONFIG_BPF_JIT_ALWAYS_ON is not set
CONFIG_BPF_JIT_DEFAULT_ON=y
CONFIG_BPF_UNPRIV_DEFAULT_OFF=y
# CONFIG_BPF_PRELOAD is not set
# end of BPF subsystem
CONFIG_CGROUP_BPF=y
CONFIG_IPV6_SEG6_BPF=y
CONFIG_NETFILTER_XT_MATCH_BPF=m
# CONFIG_BPFILTER is not set
CONFIG_NET_CLS_BPF=m
CONFIG_NET_ACT_BPF=m
CONFIG_BPF_STREAM_PARSER=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_TEST_BPF=m

And:

cat /boot/config-6.1.54 | grep XDP
CONFIG_XDP_SOCKETS=y
# CONFIG_XDP_SOCKETS_DIAG is not set

[magnus-karlsson]

Since you are not changing any of the default values, pass NULL as the configuration and see if that works. What is UMEM_SIZE in your program?

[LinArcX]

UMEM_SIZE:

#define NUM_DESCS ((XSK_RING_PROD__DEFAULT_NUM_DESCS \
            + XSK_RING_CONS__DEFAULT_NUM_DESCS) * 2)
#define UMEM_SIZE (NUM_DESCS * XSK_UMEM__DEFAULT_FRAME_SIZE)

[magnus-karlsson]

Look reasonable to me, so do not know what it is on the top of my head. Your best bet is to trace the function xdp_umem_reg() and see where it returns EINVAL. There are at least 8 checks that can give rise to this errno.

[LinArcX]

xdp_umem_reg()? Why that function?

Anyway, even passing NULL as config doesn't change anything.

I have two questions:

1. Could using a compiled 64-bit Bpf program inside my 32-bit application cause this issue? (or maybe this issue is related to the fact that my program compiled in 32-bit mode?)
2. Can running my application with a custom kernel cause this issue?

[magnus-karlsson]

xsk_umem__create() will eventually call the function xdp_umem_reg() and it has a lot of tests in it that can return EINVAL.

Your questions:

1: It is not related to any bpf program. You can use xsk_umem__create() before any bpf program comes into play. 32-bit program could be the cause though. Have not tried.

2: Yes, since I do not know what you have done to the kernel :-).

Have you tried running your program on an unmodified kernel in 64-bit mode? Just to verify that it works.

[LinArcX]

Just a question. if we use Offloaded Mode (NIC Hardware) to run xdp directly on the NIC, we still need umem area?

Since as i understood, umem is an area that will contain all packets received and the ones that are going to be sent.

So for me it makes more sence to resrever this area when we are in: Native Mode (NIC Driver) or Generic Mode (Linux Network Stack).

Correct me if i'm wrong.

(I asked this, since i want to use offloaded mode and if there is no need for umem, i can skip this function call - currently i call xdp_program__attach() with XDP_MODE_NATIVE)

[magnus-karlsson]

You always need a umem if you are using AF_XDP sockets. If you are using XDP without AF_XDP sockets, then there is no need.

I do not think you can redirect to an AF_XDP socket from a HW offloaded XDP program. Nothing that I have heard of being supported. Use a regular XDP program executed on the CPU if you want to use AF_XDP.

[LinArcX]

1. 32-bit program could be the cause though. Have not tried.

I tried this one, no problem found.

2: Yes, since I do not know what you have done to the kernel :-).

Also i tested my 32bit application on a kernel with lots of customization. still didn't see any issue regarding to umem.

The point is that i created a virtual interface in my linux machine and connect my xdp application to it. but in my main applicaiton, i connect directly to the physical NIC. maybe this cause the umem issue?

Another doubt that i have is regarding to multi-thread application. my test application is a single thread(the last part of this tutorial: https://developers.redhat.com/blog/2021/04/01/get-started-with-xdp#task_4__load_xdp_objects_with_the_custom_loader)

But my main program that i have umem error, is multi-thread.

Can this be a problem?