search

conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.31k stars 986 forks source link

[bug] editable model package duplicate includes #17223

Closed lambtonr closed 1 month ago

lambtonr commented 1 month ago

What is your question?

Hi,

I am trying to diagnose a problem with my conan recipe / project cmake. The recipe was originally a out of source recipe and working for default build but wont link to CUDA when using 'with_cuda" option.

To try and debug this i have moved the recipe to be an in source recipe, and was trying to use the conan editable tutorial from here: https://github.com/conan-io/examples2/tree/main/tutorial/developing_packages/editable_packages

conan editable list 
say/1.0
    Path: /home/rossl/repos/examples2/tutorial/developing_packages/editable_packages/say/conanfile.py
image/2.0.0
    Path: /home/rossl/repos/image/conanfile.py
 conan install .             

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[conf]
tools.system.package_manager:mode=install
tools.system.package_manager:sudo=True

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[conf]
tools.system.package_manager:mode=install
tools.system.package_manager:sudo=True

======== Computing dependency graph ========
Graph root
    conanfile.py (image/2.0.0): /home/rossl/repos/image/conanfile.py
Requirements
    libpng/1.6.39@company/stable#e778615d59f4cd9a20a8d29c83d3226f - Cache
    teklib/1.0.1@company/stable#82e6f6eb59ab3b8b21dac9357c260341 - Cache
    zlib/1.2.12@company/stable#3d5941698d16b2f18d354a109f3352c8 - Cache

======== Computing necessary packages ========
Requirements
    libpng/1.6.39@company/stable#e778615d59f4cd9a20a8d29c83d3226f:95ea6580e770571aa7aefd70f2b94298082c0dfc#b51e651fdb8a375e73ab69273f6881d2 - Cache
    teklib/1.0.1@company/stable#82e6f6eb59ab3b8b21dac9357c260341:da39a3ee5e6b4b0d3255bfef95601890afd80709#0e45d2a9e45eea1711d4631d3ebd6b00 - Cache
    zlib/1.2.12@company/stable#3d5941698d16b2f18d354a109f3352c8:b647c43bfefae3f830561ca202b6cfd935b56205#38ce19e178e7eae35a1afc08045eedaf - Cache

======== Installing packages ========
teklib/1.0.1@company/stable: Already installed! (1 of 3)
zlib/1.2.12@company/stable: Already installed! (2 of 3)
libpng/1.6.39@company/stable: Already installed! (3 of 3)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'cpp_info.names' used in: libpng/1.6.39@company/stable, zlib/1.2.12@company/stable

======== Finalizing install (deploy, generators) ========
conanfile.py (image/2.0.0): Calling generate()
conanfile.py (image/2.0.0): Generators folder: /home/rossl/repos/image/build/Release/generators
conanfile.py (image/2.0.0): CMakeToolchain generated: conan_toolchain.cmake
conanfile.py (image/2.0.0): CMakeToolchain: Preset 'conan-release' added to CMakePresets.json.
    (cmake>=3.23) cmake --preset conan-release
    (cmake<3.23) cmake <path> -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake  -DENABLE_TESTING=OFF -DENABLE_DEVELOPER_MODE=OFF -DWITH_CUDA=OFF -DWITH_OPENCV=OFF -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Release
conanfile.py (image/2.0.0): CMakeToolchain generated: /home/rossl/repos/image/build/Release/generators/CMakePresets.json
conanfile.py (image/2.0.0): CMakeToolchain generated: /home/rossl/repos/image/CMakeUserPresets.json
conanfile.py (image/2.0.0): CMakeDeps necessary find_package() and targets for your CMakeLists.txt
    find_package(teklib)
    find_package(PNG)
    target_link_libraries(... teklib::teklib PNG::PNG)
conanfile.py (image/2.0.0): Generating aggregated env files
conanfile.py (image/2.0.0): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']
Install finished successfully
Preset CMake variables:

  CMAKE_BUILD_TYPE="Release"
  CMAKE_POLICY_DEFAULT_CMP0091="NEW"
  CMAKE_TOOLCHAIN_FILE:FILEPATH="generators/conan_toolchain.cmake"
  ENABLE_DEVELOPER_MODE="OFF"
  ENABLE_TESTING="OFF"
  WITH_CUDA="OFF"
  WITH_OPENCV="OFF"

-- Using Conan toolchain: /home/rossl/repos/image/build/Release/generators/conan_toolchain.cmake
-- Conan toolchain: Setting CMAKE_POSITION_INDEPENDENT_CODE=ON (options.fPIC)
-- Conan toolchain: Defining architecture flag: -m64
-- Conan toolchain: C++ Standard 17 with extensions ON
-- Conan toolchain: Setting BUILD_SHARED_LIBS = OFF
-- Developer mode is OFF. For developement, use `-DENABLE_DEVELOPER_MODE:BOOL=ON`. Building the project for the end-user...
-- The default CMAKE_C_STANDARD used by external targets and tools is not set yet. Using the latest supported C standard that is 90
-- /usr/bin/ccache found and enabled
-- CMake-Conan: first find_package() found. Installing dependencies with Conan
-- CMake-Conan: Checking if a default profile exists
/home/rossl/.conan2/profiles/default
-- CMake-Conan: cmake_system_name=Linux
-- CMake-Conan: cmake_system_processor=x86_64
-- CMake-Conan: CMake compiler=GNU
-- CMake-Conan: CMake compiler version=11.4.0
-- CMake-Conan: [settings] compiler=gcc
-- CMake-Conan: [settings] compiler.version=11
CMake Warning at build/Release/conan_provider.cmake:296 (message):
  CMake-Conan: The C compiler is not defined.  Please define CMAKE_C_COMPILER
  or enable the C language.
Call Stack (most recent call first):
  build/Release/conan_provider.cmake:375 (append_compiler_executables_configuration)
  build/Release/conan_provider.cmake:522 (detect_host_profile)
  build/Release/_deps/_project_options-src/src/PackageProject.cmake:417 (find_package)
  build/Release/_deps/_project_options-src/src/PackageProject.cmake:433 (_property_for)
  src/CMakeLists.txt:62 (target_find_dependencies)

-- CMake-Conan: Creating profile /home/rossl/repos/image/build/Release/conan_host_profile
-- CMake-Conan: Profile: 
[settings]
arch=x86_64
os=Linux
compiler=gcc
compiler.version=11
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
build_type=Release
[conf]
tools.cmake.cmaketoolchain:generator=Unix Makefiles
tools.build:compiler_executables={"cpp":"/usr/lib/ccache/c++"}

-- CMake-Conan: Installing single configuration Release
-- CMake-Conan: conan install /home/rossl/repos/image -of=/home/rossl/repos/image/build/Release/conan --profile:host=default;--profile:host=/home/rossl/repos/image/build/Release/conan_host_profile;--profile:build=default;--build=missing

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[conf]
tools.build:compiler_executables={'cpp': '/usr/lib/ccache/c++'}
tools.cmake.cmaketoolchain:generator=Unix Makefiles
tools.system.package_manager:mode=install
tools.system.package_manager:sudo=True

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[conf]
tools.system.package_manager:mode=install
tools.system.package_manager:sudo=True

======== Computing dependency graph ========
Graph root
    conanfile.py (image/2.0.0): /home/rossl/repos/image/conanfile.py
Requirements
    libpng/1.6.39@company/stable#e778615d59f4cd9a20a8d29c83d3226f - Cache
    teklib/1.0.1@company/stable#82e6f6eb59ab3b8b21dac9357c260341 - Cache
    zlib/1.2.12@company/stable#3d5941698d16b2f18d354a109f3352c8 - Cache

======== Computing necessary packages ========
Requirements
    libpng/1.6.39@company/stable#e778615d59f4cd9a20a8d29c83d3226f:95ea6580e770571aa7aefd70f2b94298082c0dfc#b51e651fdb8a375e73ab69273f6881d2 - Cache
    teklib/1.0.1@company/stable#82e6f6eb59ab3b8b21dac9357c260341:da39a3ee5e6b4b0d3255bfef95601890afd80709#0e45d2a9e45eea1711d4631d3ebd6b00 - Cache
    zlib/1.2.12@company/stable#3d5941698d16b2f18d354a109f3352c8:b647c43bfefae3f830561ca202b6cfd935b56205#38ce19e178e7eae35a1afc08045eedaf - Cache

======== Installing packages ========
teklib/1.0.1@company/stable: Already installed! (1 of 3)
zlib/1.2.12@company/stable: Already installed! (2 of 3)
libpng/1.6.39@company/stable: Already installed! (3 of 3)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'cpp_info.names' used in: zlib/1.2.12@company/stable, libpng/1.6.39@company/stable

======== Finalizing install (deploy, generators) ========
conanfile.py (image/2.0.0): Calling generate()
conanfile.py (image/2.0.0): Generators folder: /home/rossl/repos/image/build/Release/conan/build/Release/generators
conanfile.py (image/2.0.0): CMakeToolchain generated: conan_toolchain.cmake
conanfile.py (image/2.0.0): CMakeToolchain: Preset 'conan-release' added to CMakePresets.json.
    (cmake>=3.23) cmake --preset conan-release
    (cmake<3.23) cmake <path> -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake  -DENABLE_TESTING=OFF -DENABLE_DEVELOPER_MODE=OFF -DWITH_CUDA=OFF -DWITH_OPENCV=OFF -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Release
conanfile.py (image/2.0.0): CMakeToolchain generated: /home/rossl/repos/image/build/Release/conan/build/Release/generators/CMakePresets.json
conanfile.py (image/2.0.0): CMakeToolchain generated: /home/rossl/repos/image/CMakeUserPresets.json
conanfile.py (image/2.0.0): CMakeDeps necessary find_package() and targets for your CMakeLists.txt
    find_package(teklib)
    find_package(PNG)
    target_link_libraries(... teklib::teklib PNG::PNG)
conanfile.py (image/2.0.0): Generating aggregated env files
conanfile.py (image/2.0.0): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']
Install finished successfully
-- CMake-Conan: CONAN_GENERATORS_FOLDER=/home/rossl/repos/image/build/Release/conan/build/Release/generators
-- CMake-Conan: CONANFILE=/home/rossl/repos/image/conanfile.py (image/2.0.0)
-- Conan: Target declared 'PNG::PNG'
-- CMake-Conan: find_package(ZLIB) found, 'conan install' already ran
-- Conan: Target declared 'ZLIB::ZLIB'
-- CMake-Conan: find_package(teklib) found, 'conan install' already ran
-- Conan: Target declared 'teklib::teklib'
-- CMake-Conan: find_package(Git) found, 'conan install' already ran
-- Configuring done (1.9s)
-- Generating done (0.0s)
-- Build files have been written to: /home/rossl/repos/image/build/Release

However when i do cmake --build --prefix conan-release i get the following: 

CMake Error: Could not read presets from /home/rossl/repos/image:
Duplicate preset: "conan-release"

it seems like CMakeUserPresets.json contains the following includes:

{
    "version": 4,
    "vendor": {
        "conan": {}
    },
    "include": [
        "build/Release/generators/CMakePresets.json",
        "build/Release/conan/build/Release/generators/CMakePresets.json"
    ]
}

i also end up with the duplicate generators seen here under my /build/Release/ directory: image

If i remove the "build/Release/conan/build/Release/generators/CMakePresets.json" include i can run the build and it works.

cmake --build --preset conan-release
[100%] Built target image

In addition to the editable package issue, when i try and use the editable package i get the following:

  ...
-- Conan: Target declared 'image::image'
CMake Error at build/conan/cmakedeps_macros.cmake:67 (message):
  Library 'image' not found in package.  If 'image' is a system library,
  declare it with 'cpp_info.system_libs' property

Many Thanks

Have you read the CONTRIBUTING guide?

memsharded commented 1 month ago

Hi @lambtonr

Thanks for your feedback.

I am trying to understand your setup, but I am afraid that I am missing some details.

It is not very clear how the cmake-conan is playing together with the rest, specially because I see that you are also running conan install too. If it is possible, I'd like to leave cmake-conan out of the issue, and it would be better to replicate the issue with conan install + cmake --presets commands instead. For example, the final cmake --build --prefix conan-release is difficult to understand, because if using the cmake-conan, then the Conan presets cannot be used, because they are generated after Conan is invoked, but if using cmake-conan, this happens inside CMake, so not possible.

Then, there are some details that are not clear for example the first: conan install . it is not clear in which folder this is happening, in the say package, in the image package, in neither of those? If it is done in another place, what is the conanfile that is at that place?

lambtonr commented 1 month ago

Hi @memsharded

Thanks for your reply, i completely forgot this library was using cmake-conan, i disable this the editable package work as per the example.

Im not sure if this is the best place to ask this: Now that i have the this working i have tracked down the issue to the fact the library is built with set_target_properties(image PROPERTIES CUDA_SEPARABLE_COMPILATION ON). I thought setting the same property for the test_package executable, but this didnt work. I couldn't find any exampled in conan-center-index libraries that use CUDA_SEPARABLE_COMPILATION.

This issue only occurs for shared=False builds, and only when using the library via conan, If i use 'FetchContent_Declare' to use the library it links correctly. 

/usr/bin/ld: /home/rossl/.conan2/p/b/imagea593a088b474b/p/lib/libimage.a(Filter-Blur-CUDA.cu.o): in function `__sti____cudaRegisterAll()':
tmpxft_0015c8dc_00000000-6_Filter-Blur-CUDA.cudafe1.cpp:(.text.startup+0x21): undefined reference to `__cudaRegisterLinkedBinary_4a7417d2_19_Filter_Blur_CUDA_cu_bbd7f375'
...

I dont think this is a conan issue, just not sure how to propagate this correctly for consumers.

Many Thanks

I will close this issue as the problem i originally raised is resolved.

memsharded commented 1 month ago

Thanks for the feedback and following up.

I think it might be worth creating a new ticket to check the PROPERTIES CUDA_SEPARABLE_COMPILATION ON. Indeed Conan does not have any explicit model for Cuda yet, but we are also interested in learning and improving the integrations whenever possible, so we would be interested in any feedback. As commented above the best is small reproducible projects, like starting with a simple conan new cmake_lib, tuning it with the minimal changes, then sharing it (in a zip, or in a Github repo).