[bug] `full_deploy` generates paths to `res` relative to the cache instead of the deploy folder

sykhro commented 1 year ago

Environment details

Operating System+version: Windows Server 2019
Compiler+version: Clang 15.0.7
Conan version: 2.0.5, 2.0.6
Python version: 3.11.3

Steps to reproduce

See this comment for an example command that causes the issue.

Install vulkan-loader & vulkan-headers to some output directory using the full_deploy deployer
find_package(Vulkan)

Observe that the imported target for Vulkan::Vulkan has a path which is relative to the Conan cache and not the deploy folder

Imported target "Vulkan::Vulkan" includes non-existent path

"[redacted]/full_deploy/host/vulkan-loader/1.3.239.0/Debug/x86_64/../../../vulka0e9c2dc266fb1/p/res/vulkan/registry"

in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

* The path was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and references files it does not
provide.

Logs

No response

memsharded commented 1 year ago

Hi @sykhro

Thanks for your report.

First, I'd suggest to update to 2.0.6, that comes with some bugfixes.

Then, I am trying to isolate this, tried (it seems the one at fault is the "headers", not the "loader"):

conan install --requires=vulkan-headers/1.3.243.0 --deployer=full_deploy -g CMakeDeps

But then I inspect the generated Vulkan....-data.cmake and it seems the paths are correct there, pointing to the deployed copy:

set(vulkan-headers_PACKAGE_FOLDER_RELEASE "${CMAKE_CURRENT_LIST_DIR}/full_deploy/host/vulkan-headers/1.3.243.0")
set(vulkan-headers_BUILD_MODULES_PATHS_RELEASE )

set(vulkan-headers_INCLUDE_DIRS_RELEASE "${vulkan-headers_PACKAGE_FOLDER_RELEASE}/res/vulkan/registry"
            "${vulkan-headers_PACKAGE_FOLDER_RELEASE}/include")
set(vulkan-headers_RES_DIRS_RELEASE "${vulkan-headers_PACKAGE_FOLDER_RELEASE}/res")

Can you please share with me some more details to reproduce it? Thanks!

sykhro commented 1 year ago

conan install --requires=vulkan-headers/1.3.243.0 --deployer=full_deploy -g CMakeDeps

Thanks for the quick reply! Sorry, I'm actually on 2.0.6, I tried updating before clicking "open issue" and forgot to update the post.

If I run

conan install --requires=vulkan-loader/1.3.239.0 --deployer=full_deploy -g CMakeDeps --build=missing

and look at module-Vulkan-release-x86_64-data.cmake:

set(vulkan-loader_PACKAGE_FOLDER_RELEASE "${CMAKE_CURRENT_LIST_DIR}/full_deploy/host/vulkan-loader/1.3.239.0/Release/x86_64")
set(vulkan-loader_BUILD_MODULES_PATHS_RELEASE )

set(vulkan-loader_INCLUDE_DIRS_RELEASE "${vulkan-loader_PACKAGE_FOLDER_RELEASE}/../../../vulka0e9c2dc266fb1/p/res/vulkan/registry"
            "${vulkan-loader_PACKAGE_FOLDER_RELEASE}/../../../vulka0e9c2dc266fb1/p/include")

memsharded commented 1 year ago

Could be the specific version of the package, I am not even getting that module....cmake file, let me try

No, I don't get generated the module-Vulkan file, even for that version, only the Vulkan-Headers...config.cmake file, that is weird.

memsharded commented 1 year ago

Also, it is not only the resdirs, it seems also the includedirs is the same.

memsharded commented 1 year ago

Maybe it could be related to having things in different units, Windows is bad because it doesn't allow to compute relative paths from one unit to another, maybe sharing the full logs and the location of your cache and your deployed folder.

sykhro commented 1 year ago

Maybe it could be related to having things in different units, Windows is bad because it doesn't allow to compute relative paths from one unit to another, maybe sharing the full logs and the location of your cache and your deployed folder.

My cache is in the standard path C:/Users/user/.conan2, the deployed folder is C:/source/e. Let me know if there's a way to provide a more detailed log, this is all I'm getting on -vtrace.

I'm getting it with vulkan-loader, which requires res from vulkan-headers

conan install --requires=vulkan-loader/1.3.239.0 --deployer=full_deploy -g CMakeDeps --build =missing

Full log of the comand above

```ini ======== Input profiles ======== Profile host: [settings] arch=x86_64 build_type=Release compiler=msvc compiler.cppstd=14 compiler.runtime=dynamic compiler.runtime_type=Release compiler.version=193 os=Windows [conf] Profile build: [settings] arch=x86_64 build_type=Release compiler=msvc compiler.cppstd=14 compiler.runtime=dynamic compiler.runtime_type=Release compiler.version=193 os=Windows [conf] ======== Computing dependency graph ======== vulkan-loader/1.3.239.0: Not found in local cache, looking in remotes... vulkan-loader/1.3.239.0: Checking remote: gamemaker-internal vulkan-loader/1.3.239.0: Checking remote: conancenter vulkan-loader/1.3.239.0: Downloaded recipe revision 9599fdff19ef2e78223bdd9b63e730ac vulkan-headers/1.3.239.0: Not found in local cache, looking in remotes... vulkan-headers/1.3.239.0: Checking remote: gamemaker-internal vulkan-headers/1.3.239.0: Checking remote: conancenter vulkan-headers/1.3.239.0: Downloaded recipe revision 3f678623fcd7aa3a39015b5770f4f31d Graph root cli Requirements vulkan-headers/1.3.239.0#3f678623fcd7aa3a39015b5770f4f31d - Downloaded (conancenter) vulkan-loader/1.3.239.0#9599fdff19ef2e78223bdd9b63e730ac - Downloaded (conancenter) ======== Computing necessary packages ======== Requirements vulkan-headers/1.3.239.0#3f678623fcd7aa3a39015b5770f4f31d:da39a3ee5e6b4b0d3255bfef95601890afd80709#854aca8847e06c517b5861c97603fe0c - Download (conancenter) vulkan-loader/1.3.239.0#9599fdff19ef2e78223bdd9b63e730ac:906ece26ace02323af89c7b1f712b86e52f96270 - Build ======== Installing packages ======== -------- Downloading 1 package -------- Downloading binary packages in 10 parallel threads vulkan-headers/1.3.239.0: Retrieving package da39a3ee5e6b4b0d3255bfef95601890afd80709 from remote 'conancenter' vulkan-headers/1.3.239.0: Downloading 1.8MB conan_package.tgz vulkan-headers/1.3.239.0: Decompressing 1.8MB conan_package.tgz vulkan-headers/1.3.239.0: Package installed da39a3ee5e6b4b0d3255bfef95601890afd80709 vulkan-headers/1.3.239.0: Downloaded package revision 854aca8847e06c517b5861c97603fe0c vulkan-loader/1.3.239.0: Sources downloaded from 'conancenter' vulkan-loader/1.3.239.0: Calling source() in C:\Users\Erin\.conan2\p\vulkab31c46df8e224\s\src vulkan-loader/1.3.239.0: Downloading 1.6MB sdk-1.3.239.0.tar.gz -------- Installing package vulkan-loader/1.3.239.0 (2 of 2) -------- vulkan-loader/1.3.239.0: Building from source vulkan-loader/1.3.239.0: Package vulkan-loader/1.3.239.0:906ece26ace02323af89c7b1f712b86e52f96270 vulkan-loader/1.3.239.0: Copying sources to build folder vulkan-loader/1.3.239.0: Building your package in C:\Users\Erin\.conan2\p\b\vulka4731808889187\b vulkan-loader/1.3.239.0: Calling generate() vulkan-loader/1.3.239.0: Generators folder: C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build\generators vulkan-loader/1.3.239.0: CMakeToolchain generated: conan_toolchain.cmake vulkan-loader/1.3.239.0: CMakeToolchain generated: CMakePresets.json vulkan-loader/1.3.239.0: CMakeToolchain generated: ..\..\src\CMakeUserPresets.json vulkan-loader/1.3.239.0: Generating aggregated env files vulkan-loader/1.3.239.0: Generated aggregated env files: ['conanbuild.bat', 'conanrun.bat'] vulkan-loader/1.3.239.0: Calling build() vulkan-loader/1.3.239.0: apply_conandata_patches(): No patches defined in conandata vulkan-loader/1.3.239.0: Running CMake.configure() vulkan-loader/1.3.239.0: RUN: cmake -G "Visual Studio 17 2022" -DCMAKE_TOOLCHAIN_FILE="C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/build/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="C:/Users/Erin/.conan2/p/b/vulka4731808889187/p" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" "C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\src" -- Using Conan toolchain: C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/build/generators/conan_toolchain.cmake -- Selecting Windows SDK version 10.0.22000.0 to target Windows 10.0.22621. -- The C compiler identification is MSVC 19.35.32217.1 -- The CXX compiler identification is MSVC 19.35.32217.1 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done ******************************************************************************** * NOTE: Not adding target to run update_deps.py automatically. * ******************************************************************************** -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - not found -- Found Threads: TRUE -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.39.2.windows.1") -- Looking for secure_getenv -- Looking for secure_getenv - not found -- Looking for __secure_getenv -- Looking for __secure_getenv - not found -- The ASM_MASM compiler identification is MSVC -- Found assembler: C:/Program Files/Microsoft Visual Studio/2022/Professional/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/ml64.exe -- Configuring done -- Generating done -- Build files have been written to: C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/build vulkan-loader/1.3.239.0: Running CMake.build() vulkan-loader/1.3.239.0: RUN: cmake --build "C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build" --config Release MSBuild version 17.5.1+f6fdcf537 for .NET Framework Checking Build System Building Custom Rule C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/src/loader/CMakeLists.txt asm_offset.c asm_offset.vcxproj -> C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build\loader\Release\asm_offset.exe Generating gen_defines.asm Building Custom Rule C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/src/loader/CMakeLists.txt Building Custom Rule C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/src/loader/CMakeLists.txt Assembling C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\src\loader\unknown_ext_chain_masm.asm... loader-unknown-chain.vcxproj -> C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build\loader\loader-unknown-chain.dir\Release\loader-unknown-chain.lib Building Custom Rule C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/src/loader/CMakeLists.txt dev_ext_trampoline.c phys_dev_ext.c loader-opt.vcxproj -> C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build\loader\Release\loader-opt.lib Building Custom Rule C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/src/loader/CMakeLists.txt allocation.c cJSON.c debug_utils.c extension_manual.c loader_environment.c gpa_helper.c loader.c log.c terminator.c trampoline.c unknown_function_handling.c wsi.c loader_windows.c dirent_on_windows.c Creating library C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/build/loader/Release/vulkan-1.lib and object C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/build/loader/Release/vulkan-1.exp vulkan.vcxproj -> C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build\loader\Release\vulkan-1.dll Building Custom Rule C:/Users/Erin/.conan2/p/b/vulka4731808889187/b/src/CMakeLists.txt vulkan-loader/1.3.239.0: Package '906ece26ace02323af89c7b1f712b86e52f96270' built vulkan-loader/1.3.239.0: Build folder C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build vulkan-loader/1.3.239.0: Generating the package vulkan-loader/1.3.239.0: Temporary package folder C:\Users\Erin\.conan2\p\b\vulka4731808889187\p vulkan-loader/1.3.239.0: Calling package() vulkan-loader/1.3.239.0: Running CMake.install() vulkan-loader/1.3.239.0: RUN: cmake --install "C:\Users\Erin\.conan2\p\b\vulka4731808889187\b\build" --config Release --prefix "C:/Users/Erin/.conan2/p/b/vulka4731808889187/p" -- Installing: C:/Users/Erin/.conan2/p/b/vulka4731808889187/p/lib/vulkan-1.lib -- Installing: C:/Users/Erin/.conan2/p/b/vulka4731808889187/p/bin/vulkan-1.dll vulkan-loader/1.3.239.0: package(): Packaged 1 '.dll' file: vulkan-1.dll vulkan-loader/1.3.239.0: package(): Packaged 1 '.lib' file: vulkan-1.lib vulkan-loader/1.3.239.0: package(): Packaged 1 '.txt' file: LICENSE.txt vulkan-loader/1.3.239.0: Created package revision fc254ee0226d95aa33fcf3dda8c246ad vulkan-loader/1.3.239.0: Package '906ece26ace02323af89c7b1f712b86e52f96270' created vulkan-loader/1.3.239.0: Full package reference: vulkan-loader/1.3.239.0#9599fdff19ef2e78223bdd9b63e730ac:906ece26ace02323af89c7b1f712b86e52f96270#fc254ee0226d95aa33fcf3dda8c246ad vulkan-loader/1.3.239.0: Package folder C:\Users\Erin\.conan2\p\b\vulka4731808889187\p vulkan-loader/1.3.239.0: Create VULKAN_SDK environment variable: C:\Users\Erin\.conan2\p\b\vulka4731808889187\p WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X: WARN: deprecated: 'cpp_info.filenames' used in: vulkan-headers/1.3.239.0 WARN: deprecated: 'cpp_info.names' used in: vulkan-loader/1.3.239.0, vulkan-headers/1.3.239.0 WARN: deprecated: 'env_info' used in: vulkan-loader/1.3.239.0 ======== Finalizing install (deploy, generators) ======== cli: Conan built-in full deployer to C:\source\e cli: Writing generators to C:\source\e cli: Generator 'CMakeDeps' calling 'generate()' cli: Generating aggregated env files cli: Generated aggregated env files: ['conanbuild.bat', 'conanrun.bat'] Install finished successfully ```

sykhro commented 1 year ago

I'm getting the same on conan 2.0.6, macOS 13.3.1(a) (freshly installed just for this test). In module-Vulkan-release-armv8-data.cmake, I can see the following:

set(vulkan-loader_INCLUDE_DIRS_RELEASE "${vulkan-loader_PACKAGE_FOLDER_RELEASE}/../../../vulka0e9c2dc266fb1/p/res/vulkan/registry"
            "${vulkan-loader_PACKAGE_FOLDER_RELEASE}/../../../vulka0e9c2dc266fb1/p/include")

memsharded commented 1 year ago

I am struggling to see why the module-xxxx.cmake file is being generated. Maybe some of these hints?

Are you using the latest revision from ConanCenter repo?
Is it possible that someone is doing some changes, or forcing the generation of the module-xxx.cmake file?
Are the xxxx-Config.cmake or xxx-config.cmake files being generated too? In case they are, could you please also check their contents and share it? Thanks!

sykhro commented 1 year ago

I am struggling to see why the module-xxxx.cmake file is being generated. Maybe some of these hints?

I often see those files. Are they not supposed to be there?

Are you using the latest revision from ConanCenter repo? Is it possible that someone is doing some changes, or forcing the generation of the module-xxx.cmake file?

No, I can reproduce this on a clean install on macOS with no changes whatsoever:

Local Cache
  vulkan-loader
    vulkan-loader/1.3.239.0
      revisions
        9599fdff19ef2e78223bdd9b63e730ac (2023-05-04 13:23:56 UTC)
          packages
            28d4f40e639470e6bc87a1e602c96822f089b66d
              info
                settings
                  arch: armv8
                  build_type: Release
                  compiler: apple-clang
                  compiler.version: 14
                  os: Macos
                options
                  shared: True
                requires
                  vulkan-headers/1.3.239.0#3f678623fcd7aa3a39015b5770f4f31d:da39a3ee5e6b4b0d3255bfef95601890afd80709

installed with conan install --requires=vulkan-loader/1.3.239.0 --deployer=full_deploy -g CMakeDeps --build =missing

Are the xxxx-Config.cmake or xxx-config.cmake files being generated too? In case they are, could you please also check their contents and share it?

Yes, same issue. Here's the whole folder. vulkan-loader_test.zip

Cheers!

memsharded commented 1 year ago

Thanks for the repro!

I think I have found the root cause. It doesn't seem a Conan client issue (it would be a general problem, more evident and would have hit more people), but it is very specific to the vulkan-loader recipe (if you try to use vulkan-headers only, it doesn't happen, which was weird) that contains:

def package_info():
      ...
        self.cpp_info.includedirs = self.dependencies["vulkan-headers"].cpp_info.aggregated_components().includedirs

That call to aggregated_components shouldn't be done, that is not documented, public API, and seems to be affecting the state of the cpp_info of the dependencies. I will investigate further.

sykhro commented 1 year ago

Thank you! I'll fork the recipe locally, if I end up with something that works well I'll send a PR to CCI

memsharded commented 1 year ago

I am submitting a PR for Conan 2.0.7 in https://github.com/conan-io/conan/pull/14060 that avoids the root cause, which is caching of the aggregated_components(), to avoid this potential issues. Even if it seems the recipe can be simplified, it is better to be robust against these kind of code (at the cost of some performance, but lets be robust first, performant later)

Thanks very much to you for raising this issue and all the feedback to help!

conan-io / conan