[question][1.55.0] "CMakeDeps" doesn't provide `IMPORTED_LOCATION[_<CONFIG>]` target properties for its generated Imported Targets?

hwhsu1231 commented 1 year ago

Problem Description

This issue is related to the one I posted before, which was closed with the new release version 1.55.0:

https://github.com/conan-io/conan/issues/12077

In short, the problem mentioned in the above issue is that "CMakeDeps" generator doesn't provide the following target properties for its generated Imported Targets:

However, it seems that this problem is still not solved after conducting some experiments, https://github.com/conan-io/conan/issues/12077#issuecomment-1324031251.

@memsharded @lasote

What do I miss? If I do miss somehting in the demo, can you provide a correct demonstration?

Thanks.

jcar87 commented 1 year ago

Hi @hwhsu1231 thanks for raising this question.

The targets such was Qt6::Core generated by CMakeDeps are INTERFACE IMPORTED targets, that means that they do not have the IMPORTED_LOCATION property. Internally, these targets link to actual library targets that have the IMPORTED_LOCATION_xxx.

In https://github.com/conan-io/conan/issues/12077 we made sure that the intermediate targets that do have the IMPORTED_LOCATION property correctly define it based on the imported configuration, such that consumers can map configurations as per this section in the docs.

CMakeDeps generates targets such the following:

target_link_libraries(MyApp PRIVATE Qt6::Core)

Works as expected by propagating the correct compiler and linker flags. However, the use of both generator expressions conditional on the build configuration, and interface link libraries, make them unsuitable for consumer CMake projects to make assumptions about which properties they can query. Arguably, this is how CMake targets work - but it would be great if we were provided with use cases to understand the need to query these properties from the consumer side.

hwhsu1231 commented 1 year ago

@jcar87 So what should I do to make IMPORTED_LOCATION_<CONFIG> populated?

jcar87 commented 1 year ago

@jcar87 So what should I do to make IMPORTED_LOCATION_<CONFIG> populated?

The Qt6::Core target generated by CMakeDeps is an INTERFACE IMPORTED target, that has the INTERFACE_LINK_LIBRARIES target set in such a way that it will link against the real library. As such, since the type is INTERFACE, it doesn't have an IMPORTED_LOCATION property of any kind.

As I've explained before:

target_link_libraries(MyApp PRIVATE Qt6::Core)

should work exactly the same when consuming Qt from a binary distribution of Qt, or from Conan Center. Is this not the case?

hwhsu1231 commented 1 year ago

As such, since the type is INTERFACE, it doesn't have an IMPORTED_LOCATION property of any kind.

@jcar87

The problem I'm facing is that I "cannot" use the generator expression $\<TARGET_RUNTIME_DLLS:${tgt}> in the following codes to copy DLLs of Qt6::Core to the output directory:

add_executable(main "main.cpp")

target_link_libraries(main PRIVATE Qt6::Core)

add_custom_command(TARGET main POST_BUILD
    COMMAND ${CMAKE_COMMAND} -E copy
        "$<TARGET_RUNTIME_DLLS:main>"
        "$<TARGET_FILE_DIR:main>"
    COMMAND_EXPAND_LISTS)

if the following target properties of Qt6::Core are "empty":

jcar87 commented 1 year ago

@hwhsu1231 thanks so much for providing this example.

The internal targets (generated by CMakeDeps) that would be transitively linked by Qt6::Core do have those properties set - CMake is able to walk through the dependency graph in order to resolve TARGET_RUNTIME_DLLS. So in this case, I don't think the issue has to do with Qt6::Core not having those properties.

From what I can see from the documentation here: https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#genex:TARGET_RUNTIME_DLLS

I believe the issue here would by cause by the internal targets generated by CMakeDeps not explicitly defining that the targets are SHARED. We will investigate this!

jcar87 commented 1 year ago

We may be able to provide a workaround or an alternative for this on the other hand - could you confirm which operating system and CMake generator you are using? Cheers!

hwhsu1231 commented 1 year ago

could you confirm which operating system and CMake generator you are using?

OS version: Windows 11
CMake generator:
- Visual Studio 16 2019,
- Ninja,
- Ninja Multi-Config,
- NMake Makefiles

EstebanDugueperoux2 commented 1 year ago

Any news about this issue, do someone have a workaround?

hwhsu1231 commented 1 year ago

@jcar87

According to the CMake Docs, it seems that the reason is because CMakeDeps declares those Imported Targets as INTERFACE IMPORTED. Therefore, they cannot carry the following target properties, which store the information of DLLs on disk:

I was wondering why not just declare them as:

STATIC IMPORTED for "Static" Libraries
SHARED IMPORTED for "Shared" Libraries

If I didn't understand wrong, the INTERFACE IMPORTED is designed for "Header-Only" Libraies. Is there any reason why CMakeDeps insists on declaring all the Imported Targets as INTERFACE IMPORTED?

hwhsu1231 commented 1 year ago

Take Protobuf for example.

The protobuf::libprotoc declared by Official Config Files is SHARED IMPORTED.
The protobuf::libprotoc declared by CMakeDeps is INTERFACE IMPORTED.

Screeshots:

jcar87 commented 1 year ago

Hi @hwhsu1231! Thanks for the thorough analysis. You are certainly on the right track!!

The key information is in the documentation of the TARGET_RUNTIME_DLLS generator expression.

For imported targets (as are the ones generated by CMakeDeps), this feature has 3 pre-requisites for working correctly:

that the targets are SHARED
that the IMPORTED_LOCATION property points to the DLL file
(implicitly by 2): that the IMPORTED_IMPLIB points to the .lib file.

This is not directly an issue with the protobuf::libprotoc being an "interface" target or lacking a specific property. This target, implicitly links against a target that does have the IMPORTED_LOCATION property defined. After all, this is what guarantees that if you target_link_libraries() against protobuf::libprotoc, the correct compiler and linker flags are propagated to the right places. This should also not be a problem for the TARGET_RUNTIME_DLLS feature: CMake is able to traverse the dependency graph just fine.

The issue here lies with those internal targets I mentioned earlier. The "public" target protobuf::libprotoc target, carries a dependency on an internal target generated by CMakeDeps, something along the lines of CONAN_LIB::protobuf_protobuf_libprotobuf_protobuf_RELEASE (note this target is not for public consumption and the name is not guaranteed).

The issue when it comes to TARGET_RUNTIME_DLLS is that this target does not satisfy the 3 requirements I've listed above. This target is an UKNOWN IMPORTED that has an IMPORTED_LOCATION pointing to the .lib file.

We're currently looking into this issue to see if we can ensure the behaviour is more correct.

Workarounds

In the command line

If you enable the VirtualRunEnv generator in your conanfile, it will generate a batch script that you can activate, that will amend the PATH environment variable to point to all the directories containing DLLs from your dependencies (more specifically, all bindirs specified by your dependencies). This should be enough to launch your built executables that have dependencies on DLLs from Conan dependencies. This applies when you are generating CMake projects on Windows, where the CMake generator is Ninja for instance.

In Visual Studio

If your CMake generator is the default Visual Studio generator (which will solution .sln and project .vcxproj files) you can add logic in your conanfile to copy the DLLs coming from your dependencies, into your current build directory - paying attention that these DLLs need to end up in the same location as the built executables in your build directory.

    def generate(self):

        for dep in self.dependencies.values():
            copy(self, "*.dll", dep.cpp_info.bindirs[0], os.path.join(self.build_folder, self.cpp.build.bindirs[0]))

        tc = CMakeToolchain(self)
        tc.generate()

note that you need to make sure you are using the CMakeLayout.

@EstebanDugueperoux2 - please let me know if this is a suitable workaround to get the DLLs, while we work on a long term solution that enables the use of the TARGET_RUNTIME_DLLS generator expression.

jwillikers commented 1 year ago

For shared libraries, it would help to also set the IMPORTED_SONAME property which is used when installing runtime dependency sets. It will install SO symlinks accordingly, or at least one level of symlinks according to this bug.

hwhsu1231 commented 1 year ago

@jcar87 - Should this issue be added the label type: bugs, added to the latest milestone, or related to some other issues to fix it?

flatline commented 1 year ago

I'm running into the same problem as detailed in #12957. The workaround above is only partial. It will address runtime staging of the libraries for e.g. running from an IDE, but not for the cmake install command. This information needs to somehow be available within CMake. The closest I can find is the <package>_BIN_DIRS_<CONFIG> variables generated by conan, but those appear to be semi-private conan-only variables, and rely on doing a file glob blindly.

jwillikers commented 1 year ago

@flatline I have an extensive example of using CMake to do this here. Note that this relies on the names used by Conan for its internal CMake targets, so it's not using a stable API.

famastefano commented 1 year ago

Sadly I too have this problem, so for now I'm using a workaround as I only need to install 1 .dll that is also managed by me. If anyone needs or wants to jump into the rabbit hole though, I'm posting a very useful CMake command that I've found today to dump all the properties of a target. These of course are many of the available properties of a target, but you get the idea:

include(CMakePrintHelpers)
cmake_print_properties(
    TARGETS myConanDep::myConanDep
    PROPERTIES
        IMPORTED
        IMPORTED_COMMON_LANGUAGE_RUNTIME
        IMPORTED_CONFIGURATIONS
        IMPORTED_GLOBAL
        IMPORTED_IMPLIB
        IMPORTED_IMPLIB_Debug
        IMPORTED_IMPLIB_Release
        IMPORTED_LIBNAME
        IMPORTED_LIBNAME_Debug
        IMPORTED_LIBNAME_Release
        IMPORTED_LINK_DEPENDENT_LIBRARIES
        IMPORTED_LINK_DEPENDENT_LIBRARIES_Debug
        IMPORTED_LINK_DEPENDENT_LIBRARIES_Release
        IMPORTED_LINK_INTERFACE_LANGUAGES
        IMPORTED_LINK_INTERFACE_LANGUAGES_Debug
        IMPORTED_LINK_INTERFACE_LANGUAGES_Release
        IMPORTED_LINK_INTERFACE_LIBRARIES
        IMPORTED_LINK_INTERFACE_LIBRARIES_Debug
        IMPORTED_LINK_INTERFACE_LIBRARIES_Release
        IMPORTED_LINK_INTERFACE_MULTIPLICITY
        IMPORTED_LINK_INTERFACE_MULTIPLICITY_Debug
        IMPORTED_LINK_INTERFACE_MULTIPLICITY_Release
        IMPORTED_LOCATION
        IMPORTED_LOCATION_Debug
        IMPORTED_LOCATION_Release
        IMPORTED_NO_SONAME
        IMPORTED_NO_SONAME_Debug
        IMPORTED_NO_SONAME_Release
        IMPORTED_OBJECTS
        IMPORTED_OBJECTS_Debug
        IMPORTED_OBJECTS_Release
        IMPORTED_SONAME
        IMPORTED_SONAME_Debug
        IMPORTED_SONAME_Release
        IMPORT_PREFIX
        IMPORT_SUFFIX
        INCLUDE_DIRECTORIES
        INSTALL_NAME_DIR
        INSTALL_REMOVE_ENVIRONMENT_RPATH
        INSTALL_RPATH
        INSTALL_RPATH_USE_LINK_PATH
        INTERFACE_AUTOUIC_OPTIONS
        INTERFACE_COMPILE_DEFINITIONS
        INTERFACE_COMPILE_FEATURES
        INTERFACE_COMPILE_OPTIONS
        INTERFACE_INCLUDE_DIRECTORIES
        INTERFACE_LINK_DEPENDS
        INTERFACE_LINK_DIRECTORIES
        INTERFACE_LINK_LIBRARIES
        INTERFACE_LINK_OPTIONS
        INTERFACE_POSITION_INDEPENDENT_CODE
        INTERFACE_PRECOMPILE_HEADERS
        INTERFACE_SOURCES
        INTERFACE_SYSTEM_INCLUDE_DIRECTORIES
        LIBRARY_OUTPUT_DIRECTORY
        LIBRARY_OUTPUT_DIRECTORY_Debug
        LIBRARY_OUTPUT_DIRECTORY_Release
        LIBRARY_OUTPUT_NAME
        LIBRARY_OUTPUT_NAME_Debug
        LIBRARY_OUTPUT_NAME_Release
        LINK_DEPENDS
        LINK_DEPENDS_NO_SHARED
        LINK_DIRECTORIES
        LINK_FLAGS
        LINK_FLAGS_Debug
        LINK_FLAGS_Release
        LINK_INTERFACE_LIBRARIES
        LINK_INTERFACE_LIBRARIES_Debug
        LINK_INTERFACE_LIBRARIES_Release
        LINK_INTERFACE_MULTIPLICITY
        LINK_INTERFACE_MULTIPLICITY_Debug
        LINK_INTERFACE_MULTIPLICITY_Release
        LINK_LIBRARIES
        LINK_OPTIONS
        LOCATION
        LOCATION_Debug
        LOCATION_Release
        MANUALLY_ADDED_DEPENDENCIES
        MSVC_RUNTIME_LIBRARY
        NAME
        NO_SONAME
        NO_SYSTEM_FROM_IMPORTED
        OUTPUT_NAME
        OUTPUT_NAME_Debug
        OUTPUT_NAME_Release
        PCH_WARN_INVALID
        PCH_INSTANTIATE_TEMPLATES
        PDB_NAME
        PDB_NAME_Debug
        PDB_NAME_Release
        PDB_OUTPUT_DIRECTORY
        PDB_OUTPUT_DIRECTORY_Debug
        PDB_OUTPUT_DIRECTORY_Release
        PRECOMPILE_HEADERS
        PRECOMPILE_HEADERS_REUSE_FROM
        PREFIX
        PRIVATE_HEADER
        PUBLIC_HEADER
        RESOURCE
        RUNTIME_OUTPUT_DIRECTORY
        RUNTIME_OUTPUT_DIRECTORY_Debug
        RUNTIME_OUTPUT_DIRECTORY_Release
        RUNTIME_OUTPUT_NAME
        RUNTIME_OUTPUT_NAME_Debug
        RUNTIME_OUTPUT_NAME_Release
        SOURCE_DIR
        SOURCES
        STATIC_LIBRARY_FLAGS
        STATIC_LIBRARY_FLAGS_Debug
        STATIC_LIBRARY_FLAGS_Release
        STATIC_LIBRARY_OPTIONS
        SUFFIX
        TYPE
        VERSION
)

forry commented 1 year ago

Hi, I also have a similar issue. I'm currently trying to use some kind of cmake dep. providers for conan 1. So the build is managed by using a CMake only. It runs conan install with CMakeDeps internally much like the cmake-conan for v2. Then also there is an install directive trying to copy the DLLs of the deps. I don't think the generate() workaround is applicable to me either. So a suggestion: Since the CMakeDeps provides variables like _INCLUDE_DIR it could also provide _BINARY_DIR. Then at least half of the job could not be hardcoded. The current workaround is to hardcode the binary dir to be _PACKAGE_FOLDER_RELEASE/bin and then hardcode the library name. Or in my case just use install(DIRECTORY ${<packge_name>_PACKAGE_FOLDER_RELEASE}/bin/). I suppose this could be a quick fix.

memsharded commented 1 year ago

Conan CMakeDeps is already generating a variable for binaries:

              set({{ pkg_name }}_INCLUDE_DIRS{{ config_suffix }} {{ global_cpp.include_paths }})
              ...
              set({{ pkg_name }}_LIB_DIRS{{ config_suffix }} {{ global_cpp.lib_paths }})
              set({{ pkg_name }}_BIN_DIRS{{ config_suffix }} {{ global_cpp.bin_paths }})

forry commented 1 year ago

Not in my case. What am I doing wrong? Do I need to set something special in the dependency recipe I have overlooked? In my CMakeLists.txt I have both of them coming out empty:

message("${fbxsdk_BIN_DIR}") #empty
message("${fbxsdk_BIN_DIRS}") #empty
message("${fbxsdk_BIN_DIRS_RELEASE}") #empty
message("${fbxsdk_INCLUDE_DIR}") # this one is correct

This is my case file - fbxsdk-config.cmake, generated by the conan 1.59:

########## MACROS ###########################################################################
#############################################################################################

# Requires CMake > 3.15
if(${CMAKE_VERSION} VERSION_LESS "3.15")
    message(FATAL_ERROR "The 'CMakeDeps' generator only works with CMake >= 3.15")
endif()

if(fbxsdk_FIND_QUIETLY)
    set(fbxsdk_MESSAGE_MODE VERBOSE)
else()
    set(fbxsdk_MESSAGE_MODE STATUS)
endif()

include(${CMAKE_CURRENT_LIST_DIR}/cmakedeps_macros.cmake)
include(${CMAKE_CURRENT_LIST_DIR}/fbxsdkTargets.cmake)
include(CMakeFindDependencyMacro)

check_build_type_defined()

foreach(_DEPENDENCY ${fbxsdk_FIND_DEPENDENCY_NAMES} )
    # Check that we have not already called a find_package with the transitive dependency
    if(NOT ${_DEPENDENCY}_FOUND)
        find_dependency(${_DEPENDENCY} REQUIRED ${${_DEPENDENCY}_FIND_MODE})
    endif()
endforeach()

set(fbxsdk_VERSION_STRING "2017.0.1")
set(fbxsdk_INCLUDE_DIRS ${fbxsdk_INCLUDE_DIRS_RELEASE} )
set(fbxsdk_INCLUDE_DIR ${fbxsdk_INCLUDE_DIRS_RELEASE} )
set(fbxsdk_LIBRARIES ${fbxsdk_LIBRARIES_RELEASE} )
set(fbxsdk_DEFINITIONS ${fbxsdk_DEFINITIONS_RELEASE} )

# Only the first installed configuration is included to avoid the collision
foreach(_BUILD_MODULE ${fbxsdk_BUILD_MODULES_PATHS_RELEASE} )
    message(${fbxsdk_MESSAGE_MODE} "Conan: Including build module from '${_BUILD_MODULE}'")
    include(${_BUILD_MODULE})
endforeach()

This is part of the fbxsdk recipe. I'm not setting anything special. To my knowledge, the "bin" should be a default for binary dir:

import os
from conan import ConanFile
from conan.tools.files import copy
from conans.model.version import Version

class CWExchange(ConanFile):
    name = "fbxsdk"
    version = "2017.0.1"
    settings = "os", "compiler", "arch"
    ...
    build_policy = "never"

    def package(self):
        copy(self, "*", "./include/", os.path.join(self.package_folder, "include"), keep_path=True)
        copy(self, "*.lib", "./lib/", os.path.join(self.package_folder, "lib"), keep_path=False)
        copy(self, "*.dll", "./bin/", os.path.join(self.package_folder, "bin"), keep_path=False)

    def package_info(self):
        self.cpp_info.libs = ["libfbxsdk"]

memsharded commented 1 year ago

In Conan 2.0, the requirement traits decide what is propagated and what not. For a presentation about it you can check: https://docs.conan.io/2/knowledge/videos.html (first video), and also https://docs.conan.io/2/reference/conanfile/methods/requirements.html#requirement-traits

The bindirs information will be there in case you tool_requires() an application (like cmake), or if it is a shared library (which is determined by the existence of a shared option, or by package_type = "shared-library". If fbxsdk is always a shared library, you might want to add the later.

forry commented 1 year ago

I've added the package_type = "shared-library" and re-exported the package. Then cleared the cmake cache and configured it again with no luck. Then I've added the

options = {"shared": [True, False]}
default_options = {"shared": True}

Re-exported the package, and clean configured the cmake, also no luck. I've then commented out the package_type in the cache recipe itself, and just tried to regenerate the conan deps files. And still nothing.

memsharded commented 1 year ago

I think we need a reproducible case. This is what I am trying:

from conan import ConanFile
from conan.tools.cmake import cmake_layout

class pkgRecipe(ConanFile):
    name = "pkg"
    version = "0.1"

    # Binary configuration
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False], "fPIC": [True, False]}
    default_options = {"shared": False, "fPIC": True}

    def layout(self):
        cmake_layout(self)

Then:

$ conan export-pkg . -o *:shared=True

Then, in another place:

$ conan install --requires=pkg/0.1 -o *:shared=True -g CMakeDeps

And the final file contains:

set(pkg_INCLUDE_DIRS_RELEASE "${pkg_PACKAGE_FOLDER_RELEASE}/include")
...
set(pkg_LIB_DIRS_RELEASE "${pkg_PACKAGE_FOLDER_RELEASE}/lib")
set(pkg_BIN_DIRS_RELEASE "${pkg_PACKAGE_FOLDER_RELEASE}/bin")

Maybe if you can provide something simple like the above that helps reproduce, that would be great. Thanks!

forry commented 1 year ago

Aren't you by chance using conan v2? Nevertheless, since the topic is captioned 1.55.0 I used 1.59 :)

I had one problem with cmake_layout() it complained:

pkg/0.1: Calling package()
ERROR:
        FileNotFoundError: [WinError 2] The system cannot find the file specified: 'D:\\lexocad\\conan2 test\\package\\build'

but it exported the recipe so I tried to

delete the layout() and do the rest
let the layout() there and mimic the layout by creating a build/Release folder with .lib .dll and .exp files

In both cases I needed to call the conan 1 variant for installation:

conan install pkg/0.1@ -o *:shared=True -g CMakeDeps

This is the console output of the second option (exactly your recipe) if it could be of any help:

conan export-pkg . -o *:shared=True
[HOOK - attribute_checker.py] pre_export(): WARN: Conanfile doesn't have 'url'. It is recommended to add it as attribute
[HOOK - attribute_checker.py] pre_export(): WARN: Conanfile doesn't have 'license'. It is recommended to add it as attribute
[HOOK - attribute_checker.py] pre_export(): WARN: Conanfile doesn't have 'description'. It is recommended to add it as attribute
Exporting package recipe
pkg/0.1: A new conanfile.py version was exported
pkg/0.1: Folder: D:\conan\.conan\data\pkg\0.1\_\_\export
pkg/0.1: Exported revision: cc803cf634dcbdbc2992ca2d76324219
pkg/0.1: Forced build from source
Packaging to da475edbb7454d779f78385aff6e927a8d481030
ERROR: Package already exists. Please use --force, -f to overwrite it
PS D:\lexocad\conan2 test\package> conan export-pkg . -o *:shared=True -f
[HOOK - attribute_checker.py] pre_export(): WARN: Conanfile doesn't have 'url'. It is recommended to add it as attribute
[HOOK - attribute_checker.py] pre_export(): WARN: Conanfile doesn't have 'license'. It is recommended to add it as attribute
[HOOK - attribute_checker.py] pre_export(): WARN: Conanfile doesn't have 'description'. It is recommended to add it as attribute
Exporting package recipe
pkg/0.1: The stored package has not changed
pkg/0.1: Exported revision: cc803cf634dcbdbc2992ca2d76324219
pkg/0.1: Forced build from source
Packaging to da475edbb7454d779f78385aff6e927a8d481030
pkg/0.1: Generating the package
pkg/0.1: Package folder D:\conan\.conan\data\pkg\0.1\_\_\package\da475edbb7454d779f78385aff6e927a8d481030
pkg/0.1: Calling package()
pkg/0.1: WARN: This conanfile has no package step
pkg/0.1 package(): WARN: No files in this package!
pkg/0.1: Package 'da475edbb7454d779f78385aff6e927a8d481030' created
pkg/0.1: Created package revision 1c354e0a98eac208b008e78a3cc14b2b

conan install pkg/0.1@ -o *:shared=True -g CMakeDeps
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=17
os=Windows
os_build=Windows
[options]
*:shared=True
[build_requires]
[env]

Installing package: pkg/0.1
Requirements
    pkg/0.1 from local cache - Cache
Packages
    pkg/0.1:da475edbb7454d779f78385aff6e927a8d481030 - Cache

Installing (downloading, building) binaries...
pkg/0.1: Already installed!
Generator 'CMakeDeps' calling 'generate()'
Aggregating env generators

But in both cases no binary dir in the pkg-config.cmake

I've also tried the package_type which would be the preferred way of doing this:

from conan import ConanFile
from conan.tools.cmake import cmake_layout

class pkgRecipe(ConanFile):
    name = "pkg"
    version = "0.1"

    # Binary configuration
    settings = "os", "compiler", "build_type", "arch"
    package_type = "shared-library"

    def layout(self):
        cmake_layout(self)

It generated a different package hash so it was successful but still no BINARY_DIR/S after that. Every time I have deleted the results of the install command just to be extra sure it's generated over.

memsharded commented 1 year ago

The problem is that this is not really related to the original ticket, so it was confusing to me, it is difficult to track things, and easy to assume 2.0 for new reports that are not that direct follow up from the previous conversation.

package_type, traits etc are not going to help at all for 1.X, they are only used in 2.0 resolution.

It is true that Conan 1.X is not generating the _BIN_DIRS variables in CMakeDeps, only 2.0 does it. I'd say the /bin hardcoding is a good workaround in the meantime, as bin being the bindirs folder is practically 100% guaranteed.

ericriff commented 1 year ago

I just came across this issue while trying to compile lld with Conan. CMake errors out when it hits this line https://github.com/llvm/llvm-project/blob/main/llvm/lib/WindowsManifest/CMakeLists.txt#L30 since get_property(libxml2_library TARGET LibXml2::LibXml2 PROPERTY LOCATION) returns nothing

timpatt commented 1 year ago

@jcar87, @memsharded: It seems as if there's 2 issues here:

Conan produces INTERFACE targets that then refer to the actual imported targets. This looks like a reasonable indirection to me, but the problem is that:
CMake seems to assume that the referenced target will be the actual imported library/executable/whatever in all of its DLL-related helper functions (TARGET_RUNTIME_DLLS generator expression, install IMPORTED_RUNTIME_ARTIFACTS, etc) . This seems like it could be an oversight or bug?

The question then is - should Conan produce targets that work as expected with CMake or should CMake fix this apparent oversight or both?

memsharded commented 1 year ago

Regarding the targets types, there are 2.X roadmap plans to improve the CMakeDeps to be able to generate them better. But that is a massive effort, so it will take time.

In the meantime, getting the information from self.dependencies["mydep"].cpp_info.bindirs should work pretty easily

timpatt commented 1 year ago

Thanks @memsharded. Unfortunately, my usecase (a third party tool that I'm altering) requires that Conan remain an implementation detail. I've got a couple of ideas, though...

I've raised https://gitlab.kitware.com/cmake/cmake/-/issues/24967 to capture this limitation of the CMake "runtime DLL" functionality...

memsharded commented 1 year ago

Thanks @memsharded. Unfortunately, my usecase (a third party tool that I'm altering) requires that Conan remain an implementation detail. I've got a couple of ideas, though...

One of the use case of the imported location is to collect the shared libraries of dependencies. This can be done with a --deployer=mydeploy in Conan 2.0, without modifying recipes and without modifying build scripts. It might not be optimal, in the sense that it will bring a copy of all shared libs from dependencies, not only the subset actually linked, but for many cases, it can be pretty effective.

traversebitree commented 7 months ago

Any Progress? If Conan's CMakeDeps can automatically resolve dependencies, that would be even better. Vcpkg provides two options for users to easily copy runtimes to the build directory and installation directory.

option(VCPKG_APPLOCAL_DEPS "Automatically copy dependencies into the output directory for executables." ON)

option(X_VCPKG_APPLOCAL_DEPS_INSTALL "(experimental) Automatically copy dependencies into the install target directory for executables. Requires CMake 3.14." OFF)

links:

sykhro commented 2 months ago

Friendly ping, anything I can do to help move this forward?

traversebitree commented 2 months ago

Friendly ping, anything I can do to help move this forward?

Maybe it's not an ez work 🥲

memsharded commented 2 months ago

Conan 2.4 added in https://github.com/conan-io/conan/pull/15914 the CONAN_RUNTIME_LIB_DIRS CMake variable.

This approach is the CMake recommended way to get the shared library dependencies, instead of using imported locations. The solution is documented in https://docs.conan.io/2/reference/tools/cmake/cmaketoolchain.html#conan-runtime-lib-dirs

saukijan commented 1 month ago

hi @memsharded, wouldn't install(RUNTIME_DEPENDENCY_SET) only be used when the targets are installed?

How can we copy the dependency .dll files in post-build?

memsharded commented 1 month ago

@saukijan I am not sure, that would be mostly a CMake question, not a Conan one, I don't know if it is possible or not. Maybe someone else like @jcar87 knows if this is possible?

TsXor commented 2 weeks ago

Conan 2.4 added in #15914 the CONAN_RUNTIME_LIB_DIRS CMake variable.

This approach is the CMake recommended way to get the shared library dependencies, instead of using imported locations. The solution is documented in https://docs.conan.io/2/reference/tools/cmake/cmaketoolchain.html#conan-runtime-lib-dirs

Have a question regarding this. I'm trying to use this variable but find it set to XXX/bin. The DLLs needed is put in XXX/lib so they cannot be found via this variable. After some digging in conan source code, I found: https://github.com/conan-io/conan/blob/da2b385d29271efb6b290f8721edf8a04151d256/conan/tools/cmake/toolchain/blocks.py#L610 So let me ask. Why it uses bindir on Windows? What's the purpose?

memsharded commented 2 weeks ago

HI @TsXor

So let me ask. Why it uses bindir on Windows? What's the purpose?

The convention is to put Windows "DLLs" in the "bin" folder, not the "lib" folder, because they are considered pure executables, the linker doesn't link to DLLs in Windows, only to the importing libraries (static .lib libraries), so they are pure runtime "bin", not "libs"

TsXor commented 2 weeks ago

The convention is to put Windows "DLLs" in the "bin" folder, not the "lib" folder, because they are considered pure executables, the linker doesn't link to DLLs in Windows, only to the importing libraries (static .lib libraries), so they are pure runtime "bin", not "libs"

My fault. Checked cmake doc and realized that I should use install(... RUNTIME DESTINATION bin).

RUNTIME

    Target artifacts of this kind include:

        Executables (except on macOS when marked as MACOSX_BUNDLE, see BUNDLE below);

        DLLs (on all Windows-based systems including Cygwin; note that the accompanying import libraries are of kind ARCHIVE).

TsXor commented 2 weeks ago

How can we copy the dependency .dll files in post-build?

We can use a detached cmake script. For example: In CMakeLists.txt:

if (WIN32)
    add_custom_command(
        TARGET test_package POST_BUILD
        COMMAND ${CMAKE_COMMAND}
            -D "LIBDIRS=${CONAN_RUNTIME_LIB_DIRS}"
            -D DESTDIR=$<TARGET_FILE_DIR:test_package>
            -P ${CMAKE_CURRENT_LIST_DIR}/copy-dlls.cmake
    )
endif()

Content of copy-dlls.cmake:

foreach(LIBDIR ${LIBDIRS})
    file(GLOB DLLS "${LIBDIR}/*.dll")
    foreach(DLL ${DLLS})
        execute_process(COMMAND ${CMAKE_COMMAND} -E copy -t ${DESTDIR} ${DLL})
    endforeach()
endforeach()

conan-io / conan