Access to map content without attaching bpf code

[sbernard31]

I'm a beginner with bpf/xdp, I'm currently playing with it and I plan to create a pretty simple UDP loadbalancer.

I play a bit with bcc and I feel it could make my life easier :).

But all the examples, I see is about 1 script which :

• loads bpf code
• does stuff in python/userspace by interacting with loaded bpf code (using map).
• detaches bpf code.

I would like to know if this is possible to use bcc to create several scripts :

• 1 script which could load the bpf code.
• 1 script which could update maps for configuration.
• 1 script which could monitor maps.
• 1 script which could detach bpf code (more optional because this could easily be done with iproute2).

Here is something which looks like a bit at the idea but without bcc... They compile the bpf code with clang, load it with iproute2. Then they have python scripts to monitor and configure. This scripts are using an home made bpftool wrapper. And finally they detach bpf code using iproute2 too.

Another question a bit out of topic ... Is this possible to use bcc to dump the bpf bytecode in a file and then load it later instead of recompiling C code each time. (maybe this does not make any sense because compilation is pretty fast ?)

Thx.

[yonghong-song]

For your first question, typical bcc tools are just doing tracing/trouble shooting, so one script for all makes sense.

bcc map management is different from iproutes which you needs to pin maps. typically bcc map creation happens right before prog load to avoid map relocation. bcc does support external pinned maps.

In your case, you can use bcc python or C++ APIs. Below method works for both python and C++ APIs.

• one script is to do load/attach and detach
• one script is to do monitoring (like bpftool)

C++ API contains functions for pinned maps, if you use pinned maps, you can

• one app to do load
• one app to manipulate maps
• one app to monitor (bpftool)
• one app to do detach and cleanup But I still think typically two app strategy (one for load/unload, the other for monitoring) is the best/common usage for bcc users.

bcc cannot just dump the bytecode and reload later if the map id in the bytecode becomes incorrect. But if you really have a use case, maybe you can propose an API to update map_fd in the byte code do later on users can load the function again with updated map_fd's.

[sbernard31]

@yonghong-song thx a lot.

I didn't know so much about pin map, reading this it's a bit clearer for me now.

You seems to says that only bcc C++ API provide functions for pinned maps, nothing in python right ?

But, I also understand, I can use bcc pyhton APIs to handle my loadbalancer with 2 different scripts. 1 for attach/detach and 1 for monitoring/configuration. How can I do that in python without pin map ?

Do you feel it's a good idea to use bcc for this loadbalancer use case ? Reading you, I feel this is not really thinking for that ? but maybe it makes sense to make bcc evolve a bit to support this kind of use case too ?

[yonghong-song]

It is a little bit hard to use bcc python API even to do two script thing as bcc python does not provide API for your to iterate through maps/progs in the kernel.

If python API provides pinned map support, you can have a separate python script to monitor the pinned maps. There is another tool called bpftool in kernel tree, it can dump maps/progs including map contents. https://github.com/torvalds/linux/tree/master/tools/bpf/bpftool Maybe you can explore it a little more to see whether it can fit your need.

[sbernard31]

I already played a little with bpftool and I also see some python wrapper demo.

But I really appreciate the bcc python API, so I will try more test with it. I found this, and will give it a try.

I could give you more feedbacks later, if you are interested to improve python API to support this kind of usecase.

[yonghong-song]

@sbernard31 Yes, libbpf API bpf_obj_pin() and bpf_obj-_get() are not available to python yet. If you add them to src/python/bcc/libbcc.py and add properly python interface, python can deal with pinned map/prog as well.

Yes, I am interested in how to improve python API to support your use case. Please do post in the bcc mailing list once you got some prototype/implementation. Thanks!

[alban]

Yes, libbpf API bpf_obj_pin() and bpf_obj-_get() are not available to python yet.

I would be interested to have them in python too.

I would also like to use a pinned map in bpf code. At the moment, I use bpftool with two patches:

• bpftool map exec
• bpftool map show fd 99

And I exec bpftool from python to replace the file descriptor of the map between the compilation (BPF(text=bpf_text)) and the attach (b.attach_kprobe()). Example in execsnoop