Compiling with CMAKE_USE_LIBBPG_PACKAGE results in linker error

[copyninja]

I'm trying to compile bcc using libbpf packaged on Debian (libbpf0 and libbpf-dev). So I set the above flag while invoking cmake.

mkdir build
cd build
cmake .. -DCMAKE_USE_LIBBPF_PACKAGE=1 ...

But this ends up with following error

[ 23%] Linking CXX shared library libbcc.so
cd /home/vasudeva.sk/Documents/Debian/upstream/bcc/build/src/cc && /usr/bin/cmake -E cmake_link_script CMakeFiles/bcc-shared.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC  -Wall  -fno-rtti -fPIC -DBCC_PROG_TAG_DIR='"/var/tmp/bcc"' -Wno-unused-result -DLLVM_MAJOR_VERSION=9 -O3 -DNDEBUG  -static-libstdc++  -Wl,--exclude-libs=libclangFrontend.a -Wl,--exclude-libs=libclangSerialization.a -Wl,--exclude-libs=libclangDriver.a -Wl,--exclude-libs=libclangASTMatchers.a -Wl,--exclude-libs=libclangParse.a -Wl,--exclude-libs=libclangSema.a -Wl,--exclude-libs=libclangCodeGen.a -Wl,--exclude-libs=libclangAnalysis.a -Wl,--exclude-libs=libclangRewrite.a -Wl,--exclude-libs=libclangEdit.a -Wl,--exclude-libs=libclangAST.a -Wl,--exclude-libs=libclangLex.a -Wl,--exclude-libs=libclangBasic.a  -Wl,--exclude-libs=libLLVM.a -shared -Wl,-soname,libbcc.so.0 -o libbcc.so.0.14.0 CMakeFiles/bcc-shared.dir/link_all.cc.o CMakeFiles/bcc-shared.dir/bcc_common.cc.o CMakeFiles/bcc-shared.dir/bpf_module.cc.o CMakeFiles/bcc-shared.dir/bcc_btf.cc.o CMakeFiles/bcc-shared.dir/exported_files.cc.o CMakeFiles/bcc-shared.dir/bcc_debug.cc.o CMakeFiles/bcc-shared.dir/bpf_module_rw_engine.cc.o CMakeFiles/bcc-shared.dir/table_storage.cc.o CMakeFiles/bcc-shared.dir/shared_table.cc.o CMakeFiles/bcc-shared.dir/bpffs_table.cc.o CMakeFiles/bcc-shared.dir/json_map_decl_visitor.cc.o CMakeFiles/bcc-shared.dir/bcc_syms.cc.o CMakeFiles/bcc-shared.dir/bcc_elf.c.o CMakeFiles/bcc-shared.dir/bcc_perf_map.c.o CMakeFiles/bcc-shared.dir/bcc_proc.c.o CMakeFiles/bcc-shared.dir/common.cc.o  -Wl,-rpath,/usr/lib/llvm-9/lib: frontends/b/libb_frontend.a frontends/clang/libclang_frontend.a -Wl,--whole-archive /usr/lib/llvm-9/lib/libclangFrontend.a /usr/lib/llvm-9/lib/libclangSerialization.a /usr/lib/llvm-9/lib/libclangDriver.a /usr/lib/llvm-9/lib/libclangASTMatchers.a /usr/lib/llvm-9/lib/libclangParse.a /usr/lib/llvm-9/lib/libclangSema.a /usr/lib/llvm-9/lib/libclangCodeGen.a /usr/lib/llvm-9/lib/libclangAnalysis.a /usr/lib/llvm-9/lib/libclangRewrite.a /usr/lib/llvm-9/lib/libclangEdit.a /usr/lib/llvm-9/lib/libclangAST.a /usr/lib/llvm-9/lib/libclangLex.a /usr/lib/llvm-9/lib/libclangBasic.a /usr/lib/llvm-9/lib/libLLVM-9.so.1 -Wl,--no-whole-archive -lelf libbcc_bpf.a -Wl,--whole-archive api/libapi-static.a -Wl,--no-whole-archive usdt/libusdt-static.a 
/usr/bin/ld: libbcc_bpf.a(libbpf.o): relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value

I'm not well versed with cmake to fix this can some one help me in fixing this erorr?. I'm trying to build bcc on Debian without embedding libbpf.

[yonghong-song]

cc @olsajiri could you check here? Maybe a PIC version of libbpf.a is needed here?

[copyninja]

Just to note that libbpf.a in Debian is built with -fPIC flag as seen here in the build log

[copyninja]

From reading src/cc/CMakeLists.txt here is my understanding. When CMAKE_USE_LIBBPF_PACKAGE is set bcc-shared-no-libbpf target is built but I see even bcc-shared target is built as well. bcc-shared-no-libbpf builds successfully where bcc-shared fails. So is it a build system bug?. @yonghong-song @olsajiri any comments on above?

[copyninja]

Bump.. anything on this front?

[yonghong-song]

@olsajiri Are you aware of this?

[vt-alt]

@copyninja I stumbled on the same error. Error cause is that libbpf.o for libbpf.a is compiled without -fPIC. See your Debian build log closely -- there is two compilation lines for libbpf.o:

1. cc -I. -I../include -I../include/uapi -g -O2 -fdebug-prefix-map=/<>=. -fstack-protector-strong -Wformat -Werror=format-security -Werror=implicit-function-declaration -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2 -c libbpf.c -o staticobjs/libbpf.o
2. cc -I. -I../include -I../include/uapi -g -O2 -fdebug-prefix-map=/<>=. -fstack-protector-strong -Wformat -Werror=format-security -Werror=implicit-function-declaration -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -fPIC -fvisibility=hidden -DSHARED -Wdate-time -D_FORTIFY_SOURCE=2 -c libbpf.c -o sharedobjs/libbpf.o

Only the second one (which is going into libbpf.so) is having -fPIC. The one with staticobjs/libbpf.o (which goes into libbpf.a) does not have -fPIC.

And, it seems, bcc wants libbpf.a unconditionally and wants it with -fPIC.

[copyninja]

@vt-alt right. I think this issue can be closed as it was fixed by @bluca in recent PR's and now packages in Debian built with libbpf from system