Linker error in Debug build: absl_log_internal_check_op DSO missing from command line

castle055 commented 5 months ago

I just forked the repository and was trying to setup a suitable development environment when I encountered a linker issue regarding abseil's logging library. Interestingly, this only happens when compiling the debug build.

The error in question is the following:

[100%] Linking CXX executable gnss-sdr
/usr/bin/ld: ../core/monitor/libcore_monitor.a(gnss_synchro.pb.cc.o): undefined reference to symbol '_ZN4absl12lts_2024011612log_internal17MakeCheckOpStringIPKvS4_EEPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEET_T0_PKc'
/usr/bin/ld: /usr/lib/libabsl_log_internal_check_op.so.2401.0.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make[3]: *** [src/main/CMakeFiles/gnss-sdr.dir/build.make:240: src/main/gnss-sdr] Error 1
make[2]: *** [CMakeFiles/Makefile2:2403: src/main/CMakeFiles/gnss-sdr.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:2410: src/main/CMakeFiles/gnss-sdr.dir/rule] Error 2
make: *** [Makefile:712: gnss-sdr] Error 2

This seems to indicate that the core_monitor CMake target is not linking against libabsl_log_internal_check_op.

I don't have much experience with the abseil library, so I'm not sure if this is an environment issue on my part or if this is a bug in gnss-sdr's build system. However, adding absl::log_internal_check_op within the PRIVATE scope to the target_link_libraries(core_monitor ...) call in src/core/monitor/CMakeLists.txt seems to solve the issue:

target_link_libraries(core_monitor
    PUBLIC
        Boost::headers
        Boost::system
        Gnuradio::runtime
        protobuf::libprotobuf
        core_system_parameters
    PRIVATE
        Boost::serialization
        absl::log_internal_check_op # <- Added this line
)

If this is a bug, I can submit a PR for it, but I'll need to check it doesn't conflict with any build options first because I suspect this is just a duck tape fix.

I'm compiling the next branch, specifically commit f80c537 with system provided libabsl: https://github.com/gnss-sdr/gnss-sdr/tree/f80c5373b78b5124e70d1e6743654a79145f0721

My system is running Arch Linux and abseil-cpp's binaries are installed under /usr/lib/:

$ ls -lsa /usr/lib/libabsl_log_internal_check_op*
 0 lrwxrwxrwx 1 root root    41 Apr 29 09:19 /usr/lib/libabsl_log_internal_check_op.so -> libabsl_log_internal_check_op.so.2401.0.0
36 -rwxr-xr-x 1 root root 34792 Apr 29 09:19 /usr/lib/libabsl_log_internal_check_op.so.2401.0.0

The compiler I'm using is GCC and I'm not specifying any generator to CMake, so it's using unix Makefiles.

carlesfernandez commented 5 months ago

Thanks for the detailed report (and the fix!). I really need to look into this in more detail to find out why it is happening. The core_monitor lib does not use abseil, and (if I get it right) the consumer should not link against absl::*internal* libraries. Maybe the Debug build is triggering something in Protocol Buffers?

I'll be happy to make that change if it fixes the issue, but please give me some time to check what's actually happening.

castle055 commented 5 months ago

Your are right, there seems to be an issue with protobuf since version 22 where it doesn't advertise its dependencies through the CMake module.

Here is a link to the upstream issue: https://github.com/protocolbuffers/protobuf/issues/12637

Within that thread, a comment shows a possible workaround. By using PkgConfig to manually find the needed dependencies and then linking to PkgConfig::protobuf. We would still need to call find_package(Protobuf) in order to use protobuf_generate_cpp(...) though.

Here is the link to the comment: https://github.com/protocolbuffers/protobuf/issues/12637#issuecomment-1871458639

I have tried it and it works, it figures out the abseil dependencies. But to me this doesn't sound like a great solution, since all targets would have to depend on both protobuf::libprotobufand PkgConfig::protobuf targets. Otherwise, it may not work if the build system decides to build protobuf from source instead of linking to the system library.

Maybe the best solution would be to pin protobuf to an earlier version that doesn't have this issue.

carlesfernandez commented 5 months ago

I can confirm that I can replicate the issue on Arch Linux when building in Debug mode. Interestingly, the problem does not occur when using Release, RelWithDebInfo, or even ASAN modes.

Additionally, if you remove the protobuf package and run cmake .. from a fresh start, CMake downloads and builds version 26.1 (the same version that is packaged), and this setup works fine in Debug mode. This indicates that the issue is likely related to the Arch Linux protobuf package.

I'm not very familiar with Arch Linux packaging, so I’m unable to provide a direct solution for that. However, a pragmatic approach might be to implement a more conservative (but still duck tape) fix. If in debug mode, if using abseil, if Protocol Buffers comes from a package, if the absl::log_internal_check_op library exists (bearing in mind that internal libraries might be removed in future versions based on Google's policies), and if whatever else we can narrow down, then link against absl::log_internal_check_op. Just because it works :-( .

I'm now triggering some jobs in our CI system to determine if this issue also occurs in other configurations that package recent versions of Abseil and Protocol Buffers. Thanks again for raising this issue and for providing the informative links.

castle055 commented 5 months ago

No worries.

Unfortunately, I cannot currently remove the protobuf package, as other programs depend on it. What I tried instead is to change the find_package(Protobuf) to find_package(Protobuf ${GNSSSDR_PROTOCOLBUFFERS_LOCAL_VERSION}) thinking that it would then fail to find the package and build v26.1. This works but the linker error persists.

Digging through the Makefile responsible for building the specific binary I found that it is still calling /usr/bin/protoc in order to compile the .proto files. These generated files are the ones failing to link. This leads me to believe that PROTOBUF_PROTOC_EXECUTABLE is not being set correctly in my case where a system version of protobuf is available but is older than the local version.

I can try to sandbox it inside a fake root environment or a docker container tomorrow to see if that works.

Apart from that, the problem I see with linking against absl::log_internal_check_op is that, as you said, there is no guarantee the dependency chain will remain like that for future versions. I think the PkgConfig workaround might be more robust as it avoids hard coding any library names. It still needs the same conditionals to make sure it doesn't conflict with other setups.

castle055 commented 5 months ago

Closing issue as fixed by PR #774 .

Thank you @carlesfernandez for your quick responses :smiley: