iovisor / gobpf

Go bindings for creating BPF programs.
Apache License 2.0
2.14k stars 313 forks source link

Question about mixing gobpf/bcc XDP modules and XSK maps with cgo/C AF_XDP code #203

Closed sevagh closed 5 years ago

sevagh commented 5 years ago

I'm trying to use AF_XDP. I use gobpf/bcc to load a piece of C XDP kernel code that uses redirect_map to an xsk map. It's pretty much an adaptation of https://github.com/xdp-project/xdp-tutorial/blob/master/advanced03-AF_XDP/af_xdp_kern.c except using gobpf/bcc to load the kernel code, instead of C code.

In the example I'm trying to copy, they attach the XDP module in C:

struct bpf_map *map;

bpf_obj = load_bpf_and_xdp_attach(&cfg);
if (!bpf_obj) {
    /* Error handling done in load_bpf_and_xdp_attach() */
    exit(EXIT_FAILURE);
}
/* We also need to load the xsks_map */
map = bpf_object__find_map_by_name(bpf_obj, "xsks_map");
xsks_map_fd = bpf_map__fd(map);
if (xsks_map_fd < 0) {
    fprintf(stderr, "ERROR: no xsks map found: %s\n",
        strerror(xsks_map_fd));
    exit(EXIT_FAILURE);
}

I replaced this part with Go, using gobpf/bcc:

buildArgs = []string{"-w"}
module := bcc.NewModule(source, buildArgs)
fn, _ := module.Load("my_xdp_prog", C.BPF_PROG_TYPE_XDP, 1, 131072)
module.AttachXDP("veth-xdp", fn)
xskTable := bcc.NewTable(module.TableID("my_xskmap", module)

iface, _ := net.InterfaceByName(device)
ifname_c := C.CString(iface.Name)
ifindex_c := C.int(iface.Index)

C.set_up_af_xdp_sock(ifindex_c, ifname_c) //described below

Up until here, everything is good, my XDP kernel program is loaded and attached, and the xskTable is valid and I can use TableIter() etc. with it.

After that, I call out to my C function set_up_af_xdp_sock(int ifindex, char *ifname) to set up the UMEM, AF_XDP socket, etc. Those are copied from the tutorial above, https://github.com/xdp-project/xdp-tutorial/blob/master/advanced03-AF_XDP/af_xdp_user.c#L557.

I get the error:

ERROR: Can't setup AF_XDP socket "No such file or directory"

I believe I've narrowed it down to the function xsk_socket__create, specifically the line https://github.com/libbpf/libbpf/blob/master/src/xsk.c#L579, with a combination of gdb and strace (but I'm not 100% certain).

I don't get this same error when running the example C code as-is, and the only significant difference is that:

  1. I'm loading+attaching the XDP kernel program inside Go, and then calling a C function with cgo function to set up the AF_XDP socket
  2. They're loading+attaching the XDP kernel program and setting up the AF_XDP socket all in the same C code

Thanks in advance for taking a look.

sevagh commented 5 years ago

I believe it's related to maps vs. tables.