Loading clang built eBPF objects with libbcc

[joelagnel]

Hi Guys, I am trying to use libbcc in Android to load eBPF code required for various usecases. Currently the libbcc API only supports taking a string containing C source code and building/loading that.

Some usecases in Android are building objects using clang command directly and loading it using a custom written loader, I want to be able to use libbcc to do the same so I can also use other features in libbcc such as map support, tracepoints, etc.

I could re-use some code from the Linux kernel's eBPF ELF parser from tools/lib/bpf/ but that's GPL v2.0, but Android is Apache 2.0, so this will not work.

I am thinking of spending some time to make libbcc to be able to do that (load clang built eBPF objects). Any ideas or thoughts about how to go about this? Anything to look out for? Thanks a lot.

CC: @yonghong-song @4ast @palmtenor @pchaigno @drzaeus77

[4ast]

elf loader will be a great feature addition. do you see it as C or C++ library? libbpf-static is core C libbcc-loader-static is a mix of C and C++ that was targeted as a minimal library for everything bcc minus llvm bits. If it can gain elf loader it will become usable lib for folks that don't need or want llvm in run-time.

[joelagnel]

Thanks. Yes, I was thinking of elf loader as a feature of existing libbcc library (bcc-static and bcc-shared libraries).

About addition to libbcc-loader-static, this library does not have the BCC API available along with many of the other things. I wanted perf, tracepoints, maps and other support as well minus the LLVM bits. All these things are missing in libbcc-loader-static at the moment. Also I was thinking the ELF loading itself should be made a part of the existing API in src/cc/api.

Once an elf-loader can be implemented for libbcc, one option to not include dependency on llvm/clang could be - we add build option that disables parts of BCC that need clang/llvm. Then we can just use such a libbcc without llvm/clang support and still have it fully featured (all parts of API available except for the parts that runtime-compile are available). Thoughts?

[4ast]

Majority of bcc's maps code are llvm specific. I guess some of it can be moved into bcc-loader. perf and tracepoint, I think, are already part of bcc-loader. I don't think typedef-ing everything llvm related in libbcc can work. While adding elf loader to libbcc we have to remove llvm dependency from such library. The elf loader should be part of bcc api, but library should provide either elf or llvm style loading. Projects that need both can link two libraries, but that's rare. Majority are either elf or llvm today.

[joelagnel]

Hi Alexei, thanks for your replies and the discussions. I agree with all your points, I will think some more about it as well. Perhaps we can also discuss it during LPC's eBPF micro conf.

[shwenzhang]

@joelagnel Hi joelagel，any further process？ I wonder whether libbcc is supported for Android Platform.

Have you try it successfully?

[joelagnel]

I currently run BCC inside an Android chroot environment. This works great for tracing, but I cannot use it in Android userspace for non-tracing purposes, such as data collection using tracepoint/perf/kprobes; since this is just a chroot hack. I have made some progress on that front.

libbcc itself depends on clang libraries so we don't run it on stock Android. However I have written loader to use with BCC project's libbpf. This loader is Android specific though. Ultimately, I want to move to using the libbpf mirror that Alexei created and use that, but there's still some things missing in it I think, like tracepoint creation support, last I checked.

@yonghong-song @4ast is there a plan to add the tracepoint and kprobe creation/attachment support to the libbpf mirror, like the one in BCC's libbpf ?

[yonghong-song]

@joelagnel I guess tracepoint/kprobe/uprobe creation and attachment can certainly be implemented in linux/lib/bpf/. Anybody wants to volunteer? Some selftests and samples are using kprobes/tracepoints and they can be simplified with the new API.