I think i found a bug in CMakeDeps. My setup is as follows:
Operating System: Debian GNU/Linux 12.5
Conan version 2.4.1
cmake version 3.29.5
compiler: custom made (crosstool-ng) gcc 10.3 but i think this is not relevant for the bug
The symptom of this bug is an unspecific linker failure (ld.gold fails with "internal error"). I found out, that this is caused by the sheer amount of static libs passed to the linker. In our setup most of the libs come from conan packages and contains a lot of duplicates. I believe this is a potential bug that occurs with both Ninja and Unix Makefiles CMake generators.
After trying to find the root cause i could find this particular code in cmakedeps_macros.cmake, which is one of the CMakeDeps generated files:
if(NOT TARGET ${_LIB_NAME})
# library_type can be STATIC, still UNKNOWN (if no package type available in the recipe) or SHARED (but no windows)
add_library(${_LIB_NAME} ${library_type} IMPORTED)
The condition if (NOT TARGET) should recognize already existing targets but it does not as add_library(...IMPORTED) will use directory scope, as it has not the GLOBAL parameter. In other words: The target with the very same name is created multiple times in different (directory) scopes. Adding the GLOBAL parameter to add_library will make the problem vanish. But maybe has other implications. One implication seem to be an increased time consumption for the cmake call (both, configure and generating times). The very same code appears multiple times in cmakedeps_macros.cmake but also in the other generated files for each conan package. Not sure if the other places have similar implications.
So i would appreciate if you can take a look and share your opinion on that topic.
If you need more information, please let me know. I am happy to provide it.
Thanks in advance and many greetings
Mirko
How to reproduce it
To reproduce the issue i would recommend the following:
The actual linker issue is rather hard to reproduce as our setup contains a quite large amount of code and dependencies. But there is a way to check the actual cause of this. See the minimal example here: https://github.com/mgo80/conan_cmakedeps_debug. There i tried to create a minimal cmake setup and copied the generated files from CMakeDeps into a module folder. The important part here is that at least 2 cmake calls to find_package exist in 2 different directories where none of them is a super/sub directory of the other. So i put them next to each other on the same level. And in these 2 directories libraries need to be defined which actually depend on the conan libary (here i chose openssl).
The repo has 3 branches:
master: contains the CMakeDeps generated files unchanged but you can't use it because they refer to my own conan cache directory
self_contained: Should work for anybody, especially the cmake and ninja calls from the contained README. Not sure about other OS's like Windows though
self_contained_fix: Contains an incomplete but potential fix, just for demonstration purposes
Looking at the ninja browse tool (ninja - t browse) will then show the redundant/duplicate libraries like so:
You can see the libssl.a and libcrypto.a twice here. Also i tried using the GLOBAL parameter in the mentioned cmake code in branch self_contained_fix and then the duplicates vanish:
I also think that this potential fix will not harm the link order, but maybe has other implications, which i currently don't see. At least it worked in our actual more complex environment as the actual symptom with the intern linker error did not occur anymore.
Describe the bug
I think i found a bug in CMakeDeps. My setup is as follows:
Operating System: Debian GNU/Linux 12.5 Conan version 2.4.1 cmake version 3.29.5 compiler: custom made (crosstool-ng) gcc 10.3 but i think this is not relevant for the bug
The symptom of this bug is an unspecific linker failure (ld.gold fails with "internal error"). I found out, that this is caused by the sheer amount of static libs passed to the linker. In our setup most of the libs come from conan packages and contains a lot of duplicates. I believe this is a potential bug that occurs with both Ninja and Unix Makefiles CMake generators.
After trying to find the root cause i could find this particular code in cmakedeps_macros.cmake, which is one of the CMakeDeps generated files:
The condition
if (NOT TARGET)should recognize already existing targets but it does not asadd_library(...IMPORTED)will use directory scope, as it has not theGLOBALparameter. In other words: The target with the very same name is created multiple times in different (directory) scopes. Adding theGLOBALparameter to add_library will make the problem vanish. But maybe has other implications. One implication seem to be an increased time consumption for the cmake call (both, configure and generating times). The very same code appears multiple times in cmakedeps_macros.cmake but also in the other generated files for each conan package. Not sure if the other places have similar implications.So i would appreciate if you can take a look and share your opinion on that topic.
If you need more information, please let me know. I am happy to provide it.
Thanks in advance and many greetings Mirko
How to reproduce it
To reproduce the issue i would recommend the following:
The actual linker issue is rather hard to reproduce as our setup contains a quite large amount of code and dependencies. But there is a way to check the actual cause of this. See the minimal example here: https://github.com/mgo80/conan_cmakedeps_debug. There i tried to create a minimal cmake setup and copied the generated files from CMakeDeps into a module folder. The important part here is that at least 2 cmake calls to find_package exist in 2 different directories where none of them is a super/sub directory of the other. So i put them next to each other on the same level. And in these 2 directories libraries need to be defined which actually depend on the conan libary (here i chose openssl).
The repo has 3 branches:
Looking at the ninja browse tool (ninja - t browse) will then show the redundant/duplicate libraries like so:
You can see the libssl.a and libcrypto.a twice here. Also i tried using the GLOBAL parameter in the mentioned cmake code in branch self_contained_fix and then the duplicates vanish:
I also think that this potential fix will not harm the link order, but maybe has other implications, which i currently don't see. At least it worked in our actual more complex environment as the actual symptom with the intern linker error did not occur anymore.