[BUG] The bpftime build system won't rebuild libbpf when libbpf's code changes

[agentzh]

I got bitten badly by this issue tonight. After changing code under thirdparty/libbpf/, running make build or make install won't automatically rebuild libbpf, leading to confusing results. Currently I have to make clean and then build from scratch every time I update libbpf. Can this issue in the build system be fixed? Thanks!

[Sanket-0510]

To address this issue, will need to ensure that Makefile properly tracks dependencies for libbpf so that it will trigger automatic rebuilds upon changes in thirdparty/libbpf/

what are you views @yunwei37?

[yunwei37]

We mainly use cmake for building the project, so maybe you mean use cmake to track these files?

[Officeyutong]

To address this issue, will need to ensure that Makefile properly tracks dependencies for libbpf so that it will trigger automatic rebuilds upon changes in thirdparty/libbpf/

what are you views @yunwei37?

Bpftime mainly use CMake, but libbpf doesn't support cmake. So we treat libbpf as an external project. (AddExternalProject), see https://github.com/eunomia-bpf/bpftime/blob/103903b139a9c77a7edb14866fd8a7c763385d4d/cmake/libbpf.cmake#L6

This issue might be caused by that CMake can't detect the changes in the libbpf folder, so the external project won't be rebuilt.

[NobinPegasus]

@Officeyutong I'm very new to this project. Looking ways to contribute and learn more about the codebases. I don't know if it's a bizarre idea or not. Can we store the checksum of external libbpf folder. And whenever this checksum value changes, we invoke rebuilding command?

[Officeyutong]

@Officeyutong I'm very new to this project. Looking ways to contribute and learn more about the codebases. I don't know if it's a bizarre idea or not. Can we store the checksum of external libbpf folder. And whenever this checksum value changes, we invoke rebuilding command?

libbpf store its build results (object files) under its source directory. The hash of object files might vary during different builds, so we may not use this way

[Kailian-Jacy]

To address this issue, will need to ensure that Makefile properly tracks dependencies for libbpf so that it will trigger automatic rebuilds upon changes in thirdparty/libbpf/ what are you views @yunwei37?

Bpftime mainly use CMake, but libbpf doesn't support cmake. So we treat libbpf as an external project. (AddExternalProject), see

https://github.com/eunomia-bpf/bpftime/blob/103903b139a9c77a7edb14866fd8a7c763385d4d/cmake/libbpf.cmake#L6

This issue might be caused by that CMake can't detect the changes in the libbpf folder, so the external project won't be rebuilt.

Possible solution: Makefile skips rebuilding if nothing ever changed. So the most clean way should let it decide by makefile of libbpf itself. CMake don't have to know if libbpf source code changed, just trigger make.

• If nothing changed, building process would be skipped;
• This way makes partial rebuilding possible.

Just add BUILD_ALWAYS TRUE to ExternalProject_Add(libbpf ..)

I tried on my machine to ensure it works: Before adding BUILD_ALWAYS TRUE:

1. make clean && make build under root. It compiles all of libbpf source code;
2. Add several blank space in libbpf source code, and call make build in the root of bpftime. libbpf does not recompile. Add BUILD_ALWAYS TRUE and repeat: After adding several blank space, the last modified time of that source file changes, make recognized it and recompiled:

   make[4]: Entering directory '/home/kailian/bpftime/third_party/bpftool/libbpf/src'
   make[4]: pkg-config: No such file or directory
   CC       /home/kailian/bpftime/build/libbpf/libbpf/staticobjs/str_error.o
   CC       /home/kailian/bpftime/build/libbpf/libbpf/sharedobjs/str_error.o
   INSTALL  bpf.h libbpf.h btf.h libbpf_common.h libbpf_legacy.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h skel_internal.h libbpf_version.h usdt.bpf.h
   INSTALL  /home/kailian/bpftime/build/libbpf/libbpf/libbpf.pc
   make[4]: pkg-config: No such file or directory
   CC       /home/kailian/bpftime/build/libbpf/libbpf/libbpf.so.1.3.0
   AR       /home/kailian/bpftime/build/libbpf/libbpf/libbpf.a
   INSTALL  /home/kailian/bpftime/build/libbpf/libbpf/libbpf.a /home/kailian/bpftime/build/libbpf/libbpf/libbpf.so /home/kailian/bpftime/build/libbpf/libbpf/libbpf.so.1 /home/kailian/bpftime/build/libbpf/libbpf/libbpf.so.1.3.0
   make[4]: Leaving directory '/home/kailian/bpftime/third_party/bpftool/libbpf/src'

   Rebuild triggered and only changed part recompiled.

See pr https://github.com/eunomia-bpf/bpftime/pull/278

edit. GNU make checks timestamp but not hash of source file.