Issue with libxdp and LLVM 16?

[simosund]

Hi, Recently when I tried to build the project on a testbed (or pping specifically) I ran into an issue where libxdp failed to compile (with clang/llc 16), see below.

$ ./configure 
clang: 16.0.0 (++20221208053009+4f7e5d22060e-1~exp1~20221208173118.471)
libmnl support: yes
libbpf support: submodule
ELF support: yes
zlib support: yes
libxdp support: submodule
Configuring libxdp to use our libbpf submodule
Found clang binary 'clang' with version 16 (from 'Ubuntu clang version 16.0.0 (++20221208053009+4f7e5d22060e-1~exp1~20221208173118.471)')
libbpf support: custom v1.0.0
  perf_buffer__consume support: yes
  btf__load_from_kernel_by_id support: yes
  btf__type_cnt support: yes
  bpf_object__next_map support: yes
  bpf_object__next_program support: yes
  bpf_program__insn_cnt support: yes
  bpf_map_create support: yes
  perf_buffer__new_raw support: yes
  bpf_xdp_attach support: yes
zlib support: yes
ELF support: yes
pcap support: yes
secure_getenv support: yes

$ cd pping/
$ make

  libbpf
    CC       install/lib/libbpf.a
    INSTALL  install/lib/libbpf.a

  libxdp
    CC       install/lib/libxdp.a
xdp-dispatcher.ll:10:40: warning: ptr type is only supported in -opaque-pointers mode
@dispatcher_version = dso_local global ptr null, section "xdp_metadata", align 8, !dbg !21
                                       ^
llc: error: llc: xdp-dispatcher.ll:10:40: error: expected type
@dispatcher_version = dso_local global ptr null, section "xdp_metadata", align 8, !dbg !21
                                       ^
make[4]: *** [Makefile:140: xdp-dispatcher.o] Error 1
make[3]: *** [Makefile:20: libxdp] Error 2
make[2]: *** [Makefile:28: libxdp] Error 2
make[1]: *** [Makefile:68: install/lib/libxdp.a] Error 2
make: *** [../lib/common.mk:66: ../lib/install/lib/libxdp.a] Error 2

This issue did not occur if I compiled libxdp with clang-14 instead (clang-15 was not installed on the testbed at the time). Have no really spent much time trying to figure this out yet, but thought I would report it while I still remembered.

[zzxgzgz]

Looks like we are having the same issue:

➜  bpf-examples git:(master) ✗ sudo ./configure 
clang: 15.0.6
libmnl support: yes
libbpf support: submodule
ELF support: yes
zlib support: yes
libxdp support: submodule
Configuring libxdp to use our libbpf submodule
Found clang binary 'clang' with version 15 (from 'Ubuntu clang version 15.0.6')
libbpf support: custom v1.0.0
  perf_buffer__consume support: yes
  btf__load_from_kernel_by_id support: yes
  btf__type_cnt support: yes
  bpf_object__next_map support: yes
  bpf_object__next_program support: yes
  bpf_program__insn_cnt support: yes
  bpf_map_create support: yes
  perf_buffer__new_raw support: yes
  bpf_xdp_attach support: yes
zlib support: yes
ELF support: yes
pcap support: yes
secure_getenv support: yes
➜  bpf-examples git:(master) ✗ sudo make

lib

  libxdp
    CC       install/lib/libxdp.a
llc: xdp-dispatcher.ll:10:40: error: expected type
@dispatcher_version = dso_local global ptr null, section "xdp_metadata", align 8, !dbg !21
                                       ^
make[4]: *** [xdp-dispatcher.o] Error 1
make[3]: *** [libxdp] Error 2
make[2]: *** [libxdp] Error 2
Makefile:67: recipe for target 'install/lib/libxdp.a' failed
make[1]: *** [install/lib/libxdp.a] Error 2
Makefile:34: recipe for target 'lib' failed
make: *** [lib] Error 2

[tohojo]

This usually happens if there's a version mismatch between clang and llc; can you please check config.mk and see which binaries the configure script picked up for each?

[zzxgzgz]

This usually happens if there's a version mismatch between clang and llc; can you please check config.mk and see which binaries the configure script picked up for each?

I think you're right, there's a mistach between the clang version and the llvm version:

➜  bpf-examples git:(master) ✗ which llc
/usr/bin/llc
➜  bpf-examples git:(master) ✗ llc --version
LLVM (http://llvm.org/):
  LLVM version 6.0.0

  Optimized build.
  Default target: x86_64-pc-linux-gnu
  Host CPU: skylake-avx512

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    amdgcn     - AMD GCN GPUs
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    bpf        - BPF (host endian)
    bpfeb      - BPF (big endian)
    bpfel      - BPF (little endian)
    hexagon    - Hexagon
    lanai      - Lanai
    mips       - Mips
    mips64     - Mips64 [experimental]
    mips64el   - Mips64el [experimental]
    mipsel     - Mipsel
    msp430     - MSP430 [experimental]
    nvptx      - NVIDIA PTX 32-bit
    nvptx64    - NVIDIA PTX 64-bit
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    r600       - AMD GPUs HD2XXX-HD6XXX
    sparc      - Sparc
    sparcel    - Sparc LE
    sparcv9    - Sparc V9
    systemz    - SystemZ
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
    xcore      - XCore
➜  bpf-examples git:(master) ✗  which clang
/usr/bin/clang
➜  bpf-examples git:(master) ✗ clang --version
Ubuntu clang version 15.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

I upgraded my Linux to 20.04, and re-installed clang and llvm, and I'm now able to configure/make the project.

Thank you @tohojo !

[simosund]

Can confirm that different versions of clang and llc was the issue for me as well. Apparently clang was version 16.0.0 but llc was 14.0.6. If using both clang-16 and llc-16 it compiles fine. So closing the issue. Thanks @tohojo!