search

iopsystems / rezolus

Systems performance telemetry
Other
64 stars 9 forks source link

External BTF Support #293

Open l2dy opened 4 months ago

l2dy commented 4 months ago

Expose btf_custom_path configuration from libbpf-rs to Rezolus users.

Expected behavior

Older distros or kernels built without CONFIG_DEBUG_INFO_BTF=y can be supported via external BTF files from https://github.com/aquasecurity/btfhub.

See also https://github.com/cloudflare/ebpf_exporter/blob/master/README.md#external-btf-support.

l2dy commented 4 months ago

Note for implementation:

brayniac commented 4 months ago

I'm a little unclear what you're looking for with this one. Can you go into more detail about the behavior you're expecting and how/when the user would provide the BTF files?

l2dy commented 4 months ago

The expected behavior is similar to that of ebpf_exporter's --btf.path.

Users could get the BTF files from BTFhub for supported distros, or for other distros, generate BTF files from unstripped vmlinux files by forking BTFhub. For example, kernel-debuginfo-*.rpm contains the required vmlinux file, so I wrote https://github.com/l2dy/btfgen for the extraction process.

brayniac commented 4 months ago

We currently need the BTF file at build time and it is not used at runtime. The BPF code is baked down to byte code during the build. I'm not aware of a common mechanism for providing build-time arguments in Rust projects. Currently, you could overwrite the vmlinux.h files that are in src/common/bpf/... and produce a build that works on your particular distro. I'm not sure we can offer a much better way of handling this. If you're aware of a Rust project that handles the BTF at runtime, I'd love a link.

l2dy commented 4 months ago

BTF files are loaded at runtime, that's how BPF CO-RE works.

See https://github.com/libbpf/libbpf/blob/c1a6c770c46c6e78ad6755bf596c23a4e6f6b216/src/btf.c#L5032-L5044 for the locations used by default.

brayniac commented 4 months ago

Ah. I think it clicked. (It's been a bit since I've touched the libbpf-cargo/libbpf-rs aspects).

This looks like it should be possible. I'm unlikely to have time for this for a couple of weeks. Happy to accept a PR in the meantime.