libbpf submodule

[stintel]

Is there a plan to drop the libbpf submodule, and/or to allow building bpftool against the stand-alone libbpf? Right now this requires modifications to the Makefile, making life of people trying to package bpftool harder than it needs to be. Also, the release tarball does not include libbpf, so it can't be built without extra actions.

[qmonnet]

Hi, thanks for the feedback!

Is there a plan to drop the libbpf submodule,

Not at the moment.

New bpftool features often relies on the latest additions to libbpf, so building from the master branch with a packaged version of libbpf would not always be possible. So it's easier to keep the submodule in this repo, so that folks who wish to compile bpftool don't have to worry about the libbpf version they need.

and/or to allow building bpftool against the stand-alone libbpf?

I'd be open to that, but I don't think we can at the moment. Bpftool relies on some files internal to libbpf, so we do need libbpf's sources for now.

Also, the release tarball does not include libbpf, so it can't be built without extra actions.

I had not realised, although I suppose it makes sense :thinking:. This tarball is automatically generated by GitHub, and I'm not sure I can do anything about it (I'll have a look at the settings). I can add new assets, so worst case I could add new archives that include libbpf, maybe.

[qmonnet]

Once https://github.com/libbpf/bpftool/pull/84 is merged we'll have a workflow to draft release containing binaries. It should be easy to adjust to add a new asset tarball that includes libbpf sources as well.

[qmonnet]

The PR was merged, and #85 built on top of it to add the relevant asset.

New releases now have an asset archive containing the sources for both bpftool and libbpf, see for example https://github.com/libbpf/bpftool/releases/tag/v7.2.0-snapshot.0.

[hramrach]

Nonetheless, building bpftool release against libbpf release should work so long as the internal headers are sorted out.

Maybe if bpftool needs them they are not all that internal after all?

[qmonnet]

Nonetheless, building bpftool release against libbpf release should work so long as the internal headers are sorted out.

Yes, the versions are aligned so corresponding releases for bpftool and libbpf should be synced to the same kernel commit, and it's supposed to work (cross-compiling would require two versions of the lib). That's what we do with the submodule after all.

Maybe if bpftool needs them they are not all that internal after all?

They are. They relate to various things that bpftool uses but are not part of libbpf's API. Bpftool use them because it's understood that it's tightly linked to libbpf, and that both projects are developed in the same repository so it makes sense to reuse some components. Some could possibly be part of API if you managed to convince the maintainers that there was a strong enough use case outside of bpftool; others will never be. For example, there's no way libbpf will ship its hash table implementation as part of the API.

[qmonnet]

Linking to the summary from the recent mailing list discussion on the topic of packaging bpftool, advantages and drawbacks of submodules:

https://lore.kernel.org/bpf/CAEf4Bzb9UZykRSczsP5quSEL5DvneuYfB2eLmnLKee_YrCROpw@mail.gmail.com/

As a side note, bpftool 7.2 is now available as the first release with an asset tarball containing the sources for both bpftool and libbpf.