Open maichmueller opened 11 months ago
Hi @maichmueller
Thanks for your report.
It is not fully clear what are the Conan dependencies and what are not. Is NumPy
a Conan dependency? What package depends on NumPy?
If it is not a Conan dependency but a CPM one, and it is xtensor (also a CPM dependency), then I am not fully sure how this is connected to Conan?
Please note, some dependencies (xtensor-python, xtensor-blas) are pulled in by the cmake dependency tool CPM, since they do not have a conan recipe.
It would be great to have them in Conan too. I see https://github.com/xtensor-stack/xtensor-python is a header-only library, so it should be quite easy to add it to ConanCenter too. Maybe the challenge is if it has dependencies that are not header-only and are challenging to package.
Hi @memsharded ,
sorry for the late reply!
I feared my post may be a little confusing. To clarify: The CPM dependencies are only xtensor-python
and xtensor-blas
. The former depends on NumPy
which is not a Conan dependency (there is no NumPy recipe in Conan right?), and thus on Python3 from which to grab the numpy include dirs. Python3 is also not a Conan dependency but I expect to be installed somewhere in the system for my cmake setup to work.
After investigating my issues with the migration more, I guess my main point is that cmake-conan
2 prints some weird statements that are confusing to an end-user who does not know what Conan is doing under the hood and needs to debug their setup. The confusion came from the contradictory find_package
statements which one time tell me NOT FOUND
and one time tell me FOUND
. I suppose the corresponding conan_provider.cmake
lines are these:
# Filter out `REQUIRED` from the argument list, as the first call may fail
set(_find_args "${ARGN}")
list(REMOVE_ITEM _find_args "REQUIRED")
if(NOT "MODULE" IN_LIST _find_args)
find_package(${package_name} ${_find_args} BYPASS_PROVIDER PATHS "${CONAN_GENERATORS_FOLDER}" NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH)
endif()
# Invoke find_package a second time - if the first call succeeded,
# this will simply reuse the result. If not, fall back to CMake default search
# behaviour, also allowing modules to be searched.
set(_cmake_module_path_orig "${CMAKE_MODULE_PATH}")
list(PREPEND CMAKE_MODULE_PATH "${CONAN_GENERATORS_FOLDER}")
if(NOT ${package_name}_FOUND)
find_package(${package_name} ${ARGN} BYPASS_PROVIDER)
endif()
I would argue some more explanatory messages around these consecutive find_package statements would help users debug a failing setup. Perhaps an info message like "Attempting to find package PKG_NAME only within conan generators..." ? Statements like
-- CMake-Conan: find_package(PythonInterp) found, 'conan install' already ran
-- Could NOT find PythonInterp (missing: PythonInterp_DIR) <--- python could not be found it
-- Found PythonInterp: /usr/bin/python (found version "3.8.10") <--- Here I am told it is found...
are not telling a cohesive story. Particularly, if that exact package is causing trouble when used later. More of a suggestion for future work on cmake-conan though :)
A little off-topic:
Regarding adding such CPM deps to Conan...I did try that first actually. As mentioned before, xtensor-python
needs NumPy
and Python3
which I did not understand how to bring into a conan-recipe correctly. I also found it difficult to get the main ideas from the conan tutorial in the docs. It provides a lot of detail knowledge at every step that I found mostly distracting when all I wanted to do was get a 101 example guide to correctly package a simple header-only lib. I have this problem with much of the docs (not meant to offend, I love what you guys have done with conan, the docs are merely a weaker point IMO).
This may well be my fault though. I will give it a shot again once I have more free time :+1:
Regarding the NumPy dependency, I have an open PR with a working recipe for it on CCI, if that helps: https://github.com/conan-io/conan-center-index/pull/20283 It's currently blocked by the outdated CPython package, but that can probably be substituted with an external version (which might even be preferable).
Hi, inb4: thanks a lot for your work on conan and auto-enabling it with cmake! It is well appreciated.
I am currently migrating a project from conan 1 to conan 2, but running into strange
find_package
problems.TLDR:
The cmake run with Conan2 tells me multiple times that something could not be found, then immediately tells me it was found, and eventually fails nonetheless, because it was not found after all:
Long version:
For conan 1 the project was configured with cmake-conan calling conan from within my cmake call and installing all the deps and providing paths:
I am now attempting to use
cmake-conan
as called from theCLion-Conan
plugin with conan 2. That is theconanfile.py
is default generated and my deps are inconandata.yml
withCMakeDeps
as sole generator and toolchains being created in thegenerate()
method of theconanfile.py
.I am running 2 side-by-side cmake calls, one with conan 1 and another with conan 2, to show the differences in the find_package behaviour. The find-package section of my cmake call is:
Conan 1 cmake log (important bits)
``` conanfile.txt: Generator txt created conanbuildinfo.txt ........ conanfile.txt: Generator cmake_paths created conan_paths.cmake conanfile.txt: Generator cmake created conanbuildinfo.cmake conanfile.txt: Aggregating env generators conanfile.txt: Generated conaninfo.txt conanfile.txt: Generated graphinfo -- Conan: Loading conanbuildinfo.cmake -- Conan: Adjusting output directories -- Conan: Using cmake targets configuration ........ -- Conan: Adjusting language standard -- Current conanbuildinfo.cmake directory: /home/michael/GitHub/reinforce/cmake-build-debug ........ -- Conan: Using autogenerated Findrange-v3.cmake -- Found range-v3: 0.12.0 (found version "0.12.0") -- Found Python3: /usr/bin/python3.8 (found version "3.8.10") found components: Interpreter Development NumPy Development.Module Development.Embed <---- Python3 development paths are all found without fail -- Conan: Using autogenerated Findpybind11.cmake -- Found pybind11: 2.10.4 (found version "2.10.4") ........ -- xtensor-python v0.26.1 ERRORMismatch xtensor versions. Found '..' but requires: '0.24.0' -- Found pybind11 v2.10.4 -- Found PythonInterp: /usr/bin/python (found version "3.8.10") -- Found NumPy: version "1.24.4" /home/michael/.local/lib/python3.8/site-packages/numpy/core/include <---- Numpy is found correctly by the find_package(Python3 COMPONENTS ...NumPy...) call! -- Found numpy: /home/michael/.local/lib/python3.8/site-packages/numpy/core/include <---- Numpy is found again by xtensor-python ```Conan 2 cmake log (important bits)
``` -- CMake-Conan: CONAN_GENERATORS_FOLDER=/home/michael/GitHub/reinforce/cmake-build-release/conan/build/Release/generators -- CMake-Conan: CONANFILE=/home/michael/GitHub/reinforce/conanfile.py ........... -- CMake-Conan: find_package(Python3) found, 'conan install' already ran -- Could NOT find Python3 (missing: Python3_DIR) <---- Python3 cannot be found -- Found Python3: /usr/bin/python3.8 (found version "3.8.10") found components: Interpreter Development NumPy Development.Module Development.Embed <----- Nevermind, here it is? -- CMake-Conan: find_package(pybind11) found, 'conan install' already ran -- Conan: Component target declared 'pybind11::headers' ........... -- Conan: Component target declared 'pybind11::python2_no_register' -- Conan: Target declared 'pybind11_all_do_not_use' -- Conan: Including build module from '/home/michael/.conan2/p/pybin6006d5edca4af/p/lib/cmake/pybind11/pybind11Common.cmake' ........... -- CMake-Conan: find_package(xtensor-python) found, 'conan install' already ran -- CPM: Adding package xtensor-python@0.26.1 (0.26.1) ........... -- xtensor-python v0.26.1 ........... -- Found pybind11 v2.10.4 -- CMake-Conan: find_package(NumPy) found, 'conan install' already ran -- Could NOT find NumPy (missing: NumPy_DIR) <--- Apparently numpy could no longer be found, but further down it easily can -- CMake-Conan: find_package(PythonInterp) found, 'conan install' already ran -- Could NOT find PythonInterp (missing: PythonInterp_DIR) <--- Again python could not be found it -- Found PythonInterp: /usr/bin/python (found version "3.8.10") <--- Here I am told it is found... -- Found NumPy: version "1.24.4" /home/michael/.local/lib/python3.8/site-packages/numpy/core/include <--- this is xtensor-python's find_package(NumPy REQUIRED) call. This is working fine..., but I am still getting the error of NumPy not found CMake Error at cmake-build-release/_deps/xtensor-python-src/CMakeLists.txt:58 (find_package): By not providing "FindNumPy.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "NumPy", but CMake did not find one. Could not find a package configuration file provided by "NumPy" with any of the following names: NumPyConfig.cmake numpy-config.cmake Add the installation prefix of "NumPy" to CMAKE_PREFIX_PATH or set "NumPy_DIR" to a directory containing one of the above files. If "NumPy" provides a separate development package or SDK, be sure it has been installed. ```Please note, some dependencies (
xtensor-python
,xtensor-blas
) are pulled in by the cmake dependency toolCPM
, since they do not have a conan recipe.Where do these duplicate find_package calls come from? Any advice on how to make the find_package work in conan 2 as well are well appreciated.