brainboxdotcc / DPP

C++ Discord API Bot Library - D++ is Lightweight and scalable for small and huge bots!
https://dpp.dev/
Apache License 2.0
1.01k stars 155 forks source link

Static build not working (libm linking error) #1146

Closed jonas-resch closed 16 hours ago

jonas-resch commented 2 months ago

Git commit reference Commit b39692bba42ce5577103ebb3736244d51a722eee

Describe the bug When compiling the library statically (-DBUILD_SHARED_LIBS=OFF) linking fails. (see exact error message below)

To Reproduce Steps to reproduce the behavior:

  1. Clone the repository
  2. Execute cmake -B build -DBUILD_SHARED_LIBS=OFF
  3. Execute make -C build

System Details:

Additional context

I think the issue arises when using the library in a project. Since when adding -DDPP_BUILD_TEST=OFF it works.

/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.37.a(e_logf.o): warning: relocation against_dl_x86_cpu_features' in read-only section .text' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.37.a(s_ceil.o): in functionceil_ifunc': (.text+0x2): undefined reference to _dl_x86_cpu_features' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.37.a(s_sin.o): in function__sin_ifunc': (.text+0x1222): undefined reference to _dl_x86_cpu_features' /usr/bin/ld: (.text+0x122d): undefined reference to_dl_x86_cpu_features' /usr/bin/ld: (.text+0x1236): undefined reference to _dl_x86_cpu_features' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.37.a(s_sin.o): in functioncos_ifunc': (.text+0x1282): undefined reference to _dl_x86_cpu_features' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.37.a(s_sin.o):(.text+0x128d): more undefined references to_dl_x86_cpu_features' follow /usr/bin/ld: warning: creating DT_TEXTREL in a PIE`

Mishura4 commented 1 month ago

I can check this next week

jonas-resch commented 1 week ago

Hi, just wanted to ask if you already had time to look at the issue? Thank you!

Jaskowicz1 commented 1 week ago

Can you specify exactly how you're linking D++ in your project?

jonas-resch commented 1 week ago

Hi,

I'm using cmakes FetchContent to download the library. Then I link it with target_link_libraries. These are snippets from my CMakelist.txt which are related to DPP:

FetchContent_Declare(DPP GIT_REPOSITORY "https://github.com/brainboxdotcc/DPP.git" 
    GIT_TAG "v10.0.30" OVERRIDE_FIND_PACKAGE)

FetchContent_MakeAvailable(DPP)

find_package(dpp REQUIRED)

target_link_libraries(${PROJECT_NAME}
    dpp
)

The error also occurs when I clone the DPP-repo and build it with flag BUILD_SHARED_LIBS=OFF. When I build with BUILD_SHARED_LIBS=OFF AND DPP_BUILD_TEST=OFF it works. Which means that the way the tests link DPP also has the same issue.

Let me know if you need any other information.

Jaskowicz1 commented 1 week ago

The error also occurs when I clone the DPP-repo and build it with flag BUILD_SHARED_LIBS=OFF. When I build with BUILD_SHARED_LIBS=OFF AND DPP_BUILD_TEST=OFF it works. Which means that the way the tests link DPP also has the same issue.

So, in this context, does the error occur when building D++ or does it occur when you link to that build of D++?

jonas-resch commented 1 week ago

When building DPP. So to reproduce:

Jaskowicz1 commented 1 week ago

Righto, thank you! Give me a few minutes

Jaskowicz1 commented 1 week ago

Righto, figured it out. The reason why -DDPP_BUILD_TEST=OFF fixes the issue is because the unit tests are always building dynamically, meaning some things aren't linked statically (which causes a huge freak out).

I'll have a PR up soon to fix this!

jonas-resch commented 1 week ago

Thanks a lot!

jonas-resch commented 1 week ago

Hi, in the meantime I found another issue with the static build (let me know if I should create a separate issue for it). I think zstd is not getting linked correctly (libcrypto uses libzstd). You can use this repo to reproduce the error. It already uses your previous fixes from ebdf0bd198ef046778e87ee709c852b50bb6632b.

This is the output when compiling: compile-output.txt

braindigitalis commented 1 week ago

have you tried adding the missing libraries to your target_link_libraries in the test repo? dpp isn't going to static linking to all of its dependencies and bring in openssl and all of openssls dependencies too, this would be your job wouldn't it, in your settings? it would vary from system to system and which versions of the dependencies you have available to static link against. to be honest static linking is such a rarely done thing I don't know of many using it.

jonas-resch commented 1 week ago

I tried adding the hardcoded path to zstd ( /usr/lib/x86_64-linux-gnu/libzstd.a) to the target_link_libraries in the main CMakeLists.txt file, but nothing changed. When I add /usr/lib/x86_64-linux-gnu/libzstd.a at the end of this line the Zstd errors disappear, but different libm linking errors appear.

https://github.com/brainboxdotcc/DPP/blob/c42b42cfe8ae9e8ac2c0bd4cd5056818c39cb245/library/CMakeLists.txt#L261

So that it looks like this target_link_libraries(${modname} PUBLIC ${OPENSSL_SSL_LIBRARY} ${OPENSSL_CRYPTO_LIBRARY} ${ZLIB_LIBRARIES} /usr/lib/x86_64-linux-gnu/libzstd.a)

libm linking error /usr/bin/cmake -E cmake_link_script CMakeFiles/dpp-static-build.dir/link.txt --verbose=1 /usr/bin/c++ "CMakeFiles/dpp-static-build.dir/src/main.cpp.o" -o dpp-static-build /usr/lib/x86_64-linux-gnu/libzstd.a _deps/dpp-build/library/libdpp.a -lopus -lopusfile -logg /usr/lib/x86_64-linux-gnu/libssl.a /usr/lib/x86_64-linux-gnu/libcrypto.a /usr/lib/x86_64-linux-gnu/libz.a /usr/lib/x86_64-linux-gnu/libzstd.a /usr/lib/x86_64-linux-gnu/libsodium.a /usr/lib/x86_64-linux-gnu/libopus.a /usr/lib/x86_64-linux-gnu/libm.a /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.38.a(e_logf.o): warning: relocation against `_dl_x86_cpu_features' in read-only section `.text' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.38.a(s_ceil.o): in function `__ceil_ifunc': (.text+0x2): undefined reference to `_dl_x86_cpu_features' /usr/bin/ld: /usr/lib/x86_64-linux-gnu/libm-2.38.a(e_logf.o): in function `__logf_ifunc': (.text+0x129): undefined reference to `_dl_x86_cpu_features' /usr/bin/ld: (.text+0x132): undefined reference to `_dl_x86_cpu_features' /usr/bin/ld: warning: creating DT_TEXTREL in a PIE collect2: error: ld returned 1 exit status

The only reason I need static linking is because I would like to create a .deb package for my bot. And afaik I cannot depend on a dynamically linked library which isn't in the official repos. So the only alternative would be that the user first installs DPP via the deb file and then the bot. Or do you know any other way to accomplish this?

Jaskowicz1 commented 6 days ago

The only reason I need static linking is because I would like to create a .deb package for my bot. And afaik I cannot depend on a dynamically linked library which isn't in the official repos. So the only alternative would be that the user first installs DPP via the deb file and then the bot. Or do you know any other way to accomplish this?

With this, you're best off dynamically linking D++ and letting people install it themselves. Statically linking your own means you're in charge of maintaining any D++ updates to your repo and whatnot.

jonas-resch commented 2 days ago

Thanks for the feedback. Yes, keeping the packaged library updated can be a downside. On the other hand you can guarantee that the installed version of the library is being used and no second step is required to get up and running.

I'll stick with dynamic linking for now and maybe switch to static linking when it works in the future.

Jaskowicz1 commented 2 days ago

I'll stick with dynamic linking for now and maybe switch to static linking when it works in the future.

The part that confuses me the most is why static linking doesn't work for you. For example, I statically link D++ here and it works a-okay! Granted, I disabled voice and unit tests for my static build, but it worked originally without disabling voice support!

jonas-resch commented 2 days ago

It could very well be a problem with my setup. Building the library statically on its own now works when disabling building tests. For some reason once I use it in my project via CMakes FetchContent + find_package + target_link_libraries it fails when linking.

I had a look at your example and the difference I could spot is that you are building the library separately and then linking it in the project. I am doing everything in one (downloading + building + linking). Nonetheless I couldn't get your repo to build (as by the instructions I copied the libdpp.a to the root of the repo):

Valheim-Discord-Relay/include/main.h:4:10: fatal error: dpp/dpp.h: No such file or directory

braindigitalis commented 2 days ago

It could very well be a problem with my setup. Building the library statically on its own now works when disabling building tests. For some reason once I use it in my project via CMakes FetchContent + find_package + target_link_libraries it fails when linking.

I had a look at your example and the difference I could spot is that you are building the library separately and then linking it in the project. I am doing everything in one (downloading + building + linking). Nonetheless I couldn't get your repo to build (as by the instructions I copied the libdpp.a to the root of the repo):

Valheim-Discord-Relay/include/main.h:4:10: fatal error: dpp/dpp.h: No such file or directory

youre missing -I to set the include paths. Our default build scripts and examples assume that dpp is built dynamically and installed system wide.

jonas-resch commented 17 hours ago

Thank you! Already tried that before but I messed up the path (I specified the root of the folder, not the include folder). Nonetheless I don't get why the static build works when building the library on its own, but fails when building it in a project. I will tinker a bit, maybe I can find out what's going on.

Jaskowicz1 commented 16 hours ago

Thank you! Already tried that before but I messed up the path (I specified the root of the folder, not the include folder). Nonetheless I don't get why the static build works when building the library on its own, but fails when building it in a project. I will tinker a bit, maybe I can find out what's going on.

Sounds good! I think that the problem of it failing when building in a project should be in a separate issue so, if you're able to figure out more about that and then it looks to be a D++ issue, then feel free to open a new issue! For now, I think it's safe to close this one!