Closed SeanSnyders closed 3 years ago
Hi @SeanSnyders
What if a library provide its own Config.cmake file(s)? It seems that the CMakeDeps generator does not defer to that. I would expect the generator to generate the typical files for Conan, but then also still defer/include/execute the package's original Config.cmake file(s).
Library provided config files have two limitations due to CMake:
So in principle, it is not intended that CMakeDeps can defer to such config.cmake files in the packages automatically.
A different story is that the build_modules
are not working correctly or they have some bug.
I think the issue could be that CMakeDeps
might be using only the new properties, that will replace the previous syntax of names, filenames and build_modules. Something like:
self.cpp_info.set_property("cmake_build_modules", ["mymodule.cmake"])
Please check also the new variables to activate build_modules in the build context: https://docs.conan.io/en/latest/reference/conanfile/tools/cmake/cmakedeps.html: build_context_activated, build_context_suffix, build_context_build_modules
Please have a look, we will try to reproduce from your repo too.
Hi @memsharded
The example (https://gist.github.com/SeanSnyders/30f73a76736864977bd88c0212f55fd7#file-conanfile-py-L34) I gave does enable the build modules in the build context as per the latest documentation:
But for your mention of "new properties", there is no documentation on this for cpp_info
, so nothing for me to follow:
A different story is that the build_modules are not working correctly or they have some bug. I think the issue could be that CMakeDeps might be using only the new properties, that will replace the previous syntax of names, filenames and build_modules. Something like:
self.cpp_info.set_property("cmake_build_modules", ["mymodule.cmake"])
Further investigation on the current example repro by adding below to the package_info()
method of the PureCMakePackage
external library:
self.cpp_info.set_property("cmake_build_modules", ["PureCMakePackageConfig.cmake"])
When specifying profiles on the command line (even the same default
profile for host and build), then Conan seems to include the PureCMakePackageConfig.cmake
file that is distributed/included in that external package.
This then sets the correct CMake variable for this repro.
conan install ConanCMakeDepsGenerator/0.1.0@ssnyder/testing --build=ConanCMakeDepsGenerator -s build_type=Debug --build missing --profile:host default --profile:build default
But when not specifying any profiles (which is essentially the same as specifying the same profile for host and build contexts), then Conan only notes that the target is declared but does not include the external module library.
conan install ConanCMakeDepsGenerator/0.1.0@ssnyder/testing --build=ConanCMakeDepsGenerator -s build_type=Debug --build missing
Overall this is problematic as there is no consistent way to thus include the external library into your CMake project.
The repro here of using the same default
profile is just to illustrate the same thing happening on a different project when I try to cross-compile to e.g. android using this new CMakeDeps
approach and setting the new cpp_info
property cmake_build_modules
.
Can you please try to repro on your side and respond with some further insights and direction? Thanks.
Delving deeper into Conan's source code and the CMakeDeps
generated files for this repro, I noticed the following:
The <pkg_name>_BUILD_MODULES_PATHS_<build_type>
CMake variable as generated by the Conan ConfigDataTemplate:L78
is empty when not specifying profiles (e.g. -pr:b default -pr:h default
) on the command line, and thus the build dependencies specified is not pulled in and their build modules not loaded during CMakeDeps.generate()
.
But if the build and host profile is specified, then this CMake variable contains paths to the dependency's module path as specified via cpp_info.cmake_build_modules
.
So if this is fixed, and thus the specified build modules is loaded irrespective of whether I specify profiles or not, then the main issue of this ticket should be solved.
The ConfigDataTemplate
as well as the ConfigTemplate
generates files for release
rather than e.g. the actual build_type
(e..g debug) as specified via the command line.
What if a library provide its own
*Config.cmake
file(s)
Hi @SeanSnyders
I also have a case where the conan package (hosted in a private artifactory) provides its own
In my case conan usage is restricted to a conan install, to retrieve the package, but the rest of the build is driven by calling CMake externally.
In a separate project I have implemented a completely conan free technique to get the right version of this package from the artifactory (using curl + aql) but I recently discovered the CMakeDeps and thought that might have less overhead.
Regards Baldur
@bldrvnlw Things are a bit uncertain and confusing at the moment - mainly due to lack some concrete example documentation (with toy packages as dependencies, etc) and some possible bugs in default behaviour of the feature. Maybe @memsharded can shed some more light, but through my latest experimentation in this issue I do not think it is behaving correctly yet.
I believe that setting self.cpp_info.set_property("cmake_build_modules", [<package>Config.cmake, ...]
in the package_info()
should make these available. When the host and build profiles are different (e.g. during cross-compiling) then you also need to explicitly activate your package to be available in the build context via CMakeDeps.build_context_activated = [<package>, ...]
as well as possibly add the dependency to the CMakeDeps.build_context_modules = [<package>, ...]
if you prefer to use the build context version of the dependency.
Best to check the current documentation of this feature and try things out for yourself: https://docs.conan.io/en/latest/reference/conanfile/tools/cmake/cmakedeps.html
I also have a case where the conan package (hosted in a private artifactory) provides its own config.cmake files. However in that case the package is not a build requirement but a regular requirement of the code being built. Are you saying that the only way to get this "included" is to declare it as a build requirement?
Hi @bldrvnlw
Using the internal package xxxx-config.cmake is still possible, that is only restricted in ConanCenter, because you should be aware of 2 main CMake limitations regarding xxxx-config.cmake:
find_package()
. So if you want to switch Debug/Release configuration from Visual Studio or XCode IDEs, that will not workIf despite these limitations you still want to use the internal xxx-config.cmake, you can do it, by avoiding the CMakeDeps
generator altogether, as you don't want Conan to generate the xxxx-config.cmake files for the dependencies. The CMakeToolchain
might still be useful and it is intended to allow locating the xxx-config.cmake inside the packages too.
This discussion is mostly about the extra cmake modules that the CMakeDeps
generator will be injecting, which is a different issue than wanting to use your internal xxx-config.cmake modules.
Please let me know if this helps.
@SeanSnyders yes, we know it is still not very clear, we are still heavily working and changing things in all this area, that is the reason the docs are still scarce and confusing. We are trying to stabilize and provide full examples (like https://github.com/conan-io/examples/pull/90)
My use case was very similar and actually used to work... in conan 1.36, but no longer afterward.
I built a "Qt5" binary package to put together our own special build of Qt. It's very simple and doesn't describe any components on its own, but rather aims at delegating all the details to the existing lib/cmake/Qt5/Qt5Config.cmake
provided by Qt which is the entry point for hundreds of additional .cmake files...
I did it that way:
def package_info(self):
qt5config = os.path.join('lib', 'cmake', 'Qt5', 'Qt5Config.cmake')
self.cpp_info.set_property('cmake_build_modules', [qt5config])
This used to work fine with 1.36, as this build module was loaded and thanks to that, all the logic for the various components was available.
However, since 1.37, the build modules are loaded last, so the check for components performed earlier in the generated Qt5Targets.cmake
will now fail:
...
Conan: Component 'Core' NOT found in package 'Qt5'
Call Stack (most recent call first):
build/Qt5Targets.cmake:35 (conan_message)
build/Qt5Config.cmake:9 (include)
...
So my request is the same as Sean's: it would be really great if there was an easy way (like we had in 1.36) to reuse the sometimes complex logic of existing cmake files, as provided by the targeted packages. If there's no particular reason to include the build modules last as it's done now, it would be great to simply move that up. If not, perhaps an alternative group of build modules (silly name: cmake_pre_build_modules
?) could be defined and these ones could be included prior to the generated *Targets.cmake
file?
Hi @SeanSnyders, I've been looking at this, the thing is, the new toolchain generators like CMakeDeps
and CMakeToolchain
or the ones for msbuild
, meson
, bazel
, autotools
... all the new generators that work with the generate(self)
method, are programmed assuming that the build profile is always specified. That is because, in the Conan 2.0 model, the build profile will be always present (declared or not by the user). That default behavior cannot be done in the 1.X because it would be breaking behavior (because without specifying it, all the build requirements are in the host context and not in the build context) but these new generators are there to help to migrate the recipes to the 2.0 model.
So, I agree the documentation is not explaining that correctly. What we are going to do:
A) Modify the documentation with a note
indicating that the usage of build_context_activated, build_context_suffix and build_context_build_modules
will fail if no build profile is specified.
B) Will raise when using these properties and there is no build profile
C) Every new generator will print a warning in the constructor if no build profile is specified.
So, summarizing, you would need to specify the build profile, sorry.
@cboos your issue is totally different. We are evaluating the best way to keep your use case working. I'll report back.
@cboos we are going to move the check later and also will check that the actual complete target exists, not based on the internal XXX_COMPONENT_NAMES
variable so probably it will work smoothly for you, if the Qt build module declares the target, it would be fine. It will be released at 1.39
Hi @lasote Thanks for the explanation, this is now indeed making more sense into why it is failing for certain scenarios and not for others. I'll experiment with that. That can explain point 1 in https://github.com/conan-io/conan/issues/9112#issuecomment-865511576, what of point 2?
Sure. 1=> is a consequence of the missing build profile, the build modules are not assigned. 2=> Not sure about the mixed build type, probably you have installed both release and debug configurations. Take into account, as marks a comment in the generated config file, that the build_modules are taken from the first configuration installed, that's because there is no CMake multi config mechanism for the build modules, they are just "includes".
Hi @lasote
I'd like to get back to point 2 from my comment https://github.com/conan-io/conan/issues/9112#issuecomment-865511576: Release being generated when Debug build_type is specified for dependencies.
I am having issues with this where I specify Debug and have GTest/1.10.0 as a (build) dependency and, as in the screenshot in the previous comment, it generates a mixed situation for GTest. This causes linking failure when trying to link a debug unit test app with the release version of GTest that is picked up / generated by Conan:
Running command (even explicitly setting Debug for GTest):
conan install . -o shared=False -s build_type=Debug -if=cmake-build-debug -s gtest:build_type=Debug -e CONAN_RUN_TESTS=1 --profile:host=default --profile:build=default --build=missing
Shows these settings
Configuration (profile_host):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=Visual Studio
compiler.runtime=MDd
compiler.version=16
os=Windows
os_build=Windows
gtest:build_type=Debug
[options]
shared=False
[build_requires]
[env]
CONAN_RUN_TESTS=1
Configuration (profile_build):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=16
os=Windows
os_build=Windows
[options]
[build_requires]
[env]
and these requirements
Requirements
Python requires
xxxxxx/0.5.0@xxxxxxxx/testing
Packages
Build requirements
xxxxxxxx/0.5.0@xxxxxxx/testing from 'xxxxxxxxxx' - Cache
cmake/3.19.4 from 'conan-center' - Cache
gtest/1.10.0 from 'conan-center' - Cache
Build requirements packages
xxxxxxxxxx/0.5.0@xxxxxx/testing:yyyyyyyyyyyyyyy- Cache
cmake/3.19.4:0a420ff5c47119e668867cdb51baff0eca1fdb68 - Cache
gtest/1.10.0:5ad274d83035c78ba2b205e6cf4f1b317aee8e05 - Cache
and note that the package ID for GTest is the Release version even though there is a Debug version on conan-center (https://conan.io/center/gtest?version=1.10.0&revision=5ee038e1d39b4057666e77e49011079e&tab=configuration&os=Windows) for the package.
I am confused as to the situation that the GTest conan package contains a Debug version and Conan knows about the different configurations, and the build_type is Debug, so why does Conan not generate cmake files pointing to the right binary package?
How should this work? How do I get my situation to work?
If I read correctly, Conan's build_modules is picking up the first installed configuration of GTest.
So, I did conan remove gtest
before I did the same conan install
command above with build_type=Debug
, and still conan downloaded the Release version of GTest (gtest/1.10.0:5ad274d83035c78ba2b205e6cf4f1b317aee8e05
)
Looking closer at the output, the profile_build
still notes the build_type
as Release
. So the command line argument -s build_type=Debug
only got set on the host
profile. This causes the issue.
I created a specific debug profile file that just contains the build_type
as Debug
and then added it as another profile to be used for both host and build and then conan downloads and uses the correct version of GTest, e.g.
-pr:h default -pr:h buildtype_debug -pr:b default -pr:b buildtype_debug
or specifying as command line arguments directly -s:b build_type=Debug -s:h build_type=Debug
Is this a bug? Or what is the logic here?
Environment Details (include every applicable attribute)
Problem
The new
CMakeDeps
generator behaviour generates generic CMake*Config.cmake
etc files from the Conan recipes for the external libraries that are specified as dependencies. What if a library provide its own*Config.cmake
file(s)? It seems that theCMakeDeps
generator does not defer to that. I would expect the generator to generate the typical files for Conan, but then also still defer/include/execute the package's original*Config.cmake
file(s).Also, if there is any bootstrapping that needs to occur for said external library and this logic is embedded and shipped with the external library's package, how should that be accessed /executed using the
CMakeDeps
generator. For past generators likefind_package
, it generates aFind<packagename>.cmake
file, but this still had the functionality to include the build module if specified viaself.cpp_info.build_modules[]
in the recipe methodpackage_info()
for the library. Addingself.cpp_info.build_modules["CMakeDeps"].append(...)
does not seem to make any difference for theCMakeDeps
generator.Steps to reproduce
Step 1
Clone this repo to get a Conan package (in a sub-folder of this repo) that only contains CMake files that you will install into your local cache. https://github.com/SeanSnyders/TestExamples.git
Step 2
Then use the gist below that is a recipe that depends on the package above that you installed. This recipe uses the Conan helloworld example as source code, and patches it to suit this test. https://gist.github.com/SeanSnyders/30f73a76736864977bd88c0212f55fd7
Try to build this package for this gist and it'll fail because it cannot find a CMake utility file that it wants to include that is part of the first package above.
It fails because to illustrate the problem, this dependent library sets a variable
CMAKE_PURECMAKEPACKAGE_PACKAGE_PATH
which is defined in the library's own CMake config file. Conan does not seem to include this, thus this variable is not defined, and thus the 2nd package could not be built.Logs (Executed commands with output)
Step 1
Step 2
Download gist specified above into new folder, then
Then build and install this package into the local cache