[question] Conan editables with multilibs packages

Todiq commented 1 year ago

What is your question?

Hello,

I am trying to understand how to use conan editables with packages containing multiple shared libraries. I don't encounter any issue with a single-package library. I presume it comes from a misusage from my side of the layout(self) method, since it works if the package is fully create via conan create. I tried following the instructions here and here, but still got unlucky.

I am using Rocky 8.7 with GCC 8.5.0 and Python 3.9.13 Here is my profile

[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=8
os=Linux

[conf]
tools.build:skip_test=True

and here is my layout method

def layout(self):
    # cmake_layout(self)
    self.folders.source = "."
    self.folders.build = Path("build") / str(self.settings.build_type)
    self.folders.generators = Path(self.folders.build) / "generators"

    self.cpp.source.includedirs = ["."]
    self.cpp.build.libdirs = ["."]

    for compname, comp in self._parsing_component_tree.items():
        if comp.is_lib is True:
            self.cpp.package.libs += [f"{compname}"]
            self.cpp.build.libs += [f"{compname}"]

with a small part of the _parsing_component_tree :

_ParsingComponent = namedtuple("_ParsingComponent", ("dependencies", "external_dependencies", "is_lib"))
_parsing_component_tree = {
    "parsing": _ParsingComponent([], [], False),
    "lexparse": _ParsingComponent([], [], False),
    "a3sqlu": _ParsingComponent(["pkg1", "pkg2"], ["ext_pkg1::extpkg1", "ext_pkg2::extpkg2"], True)
}

When set as editable and used by another recipe, I get the following error (small portion of the output), with AAA being the current recipe's name, BBB being the previous one (conanfile above) and XXX being the missing library:

CMake Error at build/Debug/generators/cmakedeps_macros.cmake:66 (message):
  Library 'XXX' not found in package.  If 'XXX' is a system library,
  declare it with 'cpp_info.system_libs' property
Call Stack (most recent call first):
  build/Debug/generators/BBB-Target-debug.cmake:24 (conan_package_library_targets)
  build/Debug/generators/BBBTargets.cmake:26 (include)
  build/Debug/generators/BBB-config.cmake:16 (include)
  LibraryOfAAA/CMakeLists.txt:7 (find_package)

-- Configuring incomplete, errors occurred!
ERROR: conanfile.py (AAA/1.0): Error in build() method, line 124
        cmake.configure()
        ConanException: Error 1 while executing

Hopefully you can give me a hand. Please let me know if you need more context. Thanks in advance.

Have you read the CONTRIBUTING guide?

[x] I've read the CONTRIBUTING guide

memsharded commented 1 year ago

Hi @Todiq

I'd need to have a closer look, but some quick hints:

Conan always uses old-style paths, instead of Pathlib ones. When defining folders like folders.source it should receive old style paths
I think the components are missing the libdirs, bindirs, etc. The moment components are defined, the global cpp_info is discarded, and only components information is used. You might need to define self.cpp_info.components["comp"].xxxxdirs = .. too

Todiq commented 1 year ago

Hi @memsharded,

Thank you for your time and patience. Regarding your advices : I noted that I should use os.join.path instead of Pathlib. I also see that self.cpp_info.components["comp"].xxxxdirs are not defined in my recipe (as you can see below), which can lead to the problem.

I can give you my package_info(self) method (taken from poco), which has worked flawlessly until now for full fledged packages. I used to accompany it with cmake_layout, but since I have an unusual source tree, that would not work with editables. I also tried with basic_layout(self), but no luck. That's why I am now trying to define my custom layout.

def package_info(self):
    self.cpp_info.set_property("cmake_file_name", self.name)
    self.cpp_info.set_property("cmake_target_name", f"{self.name}::{self.name}")

    for compname, comp in self._parsing_component_tree.items():
        conan_component = f"{self.name}_{compname.lower()}"
        requires = [f"{self.name}_{dependency.lower()}" for dependency in comp.dependencies] + comp.external_dependencies
        self.cpp_info.components[conan_component].set_property("cmake_target_name", f"{self.name}::{compname}")
        self.cpp_info.components[conan_component].set_property("cmake_file_name", compname)
        if comp.is_lib:
            self.cpp_info.components[conan_component].libs = [f"{compname}"]
        self.cpp_info.components[conan_component].requires = requires

memsharded commented 1 year ago

Thanks for sharing. Then yes, I think definitely the first step is to add there the definition of xxxdirs for all components (typically the includedirs would be different, but the libdirs could be common to all of them, but that depends on the project). Please try that and let us know.

Todiq commented 1 year ago

Hello @memsharded,

Unfortunately, even with your advices, I could not make it work. I had a look at the file mentionned in the error: build/Debug/generators/BBB-Target-debug.cmake:24 (conan_package_library_targets). And I indeed noticed that the library cannot be found here:

set_property(TARGET BBB_DEPS_TARGET
             PROPERTY INTERFACE_LINK_LIBRARIES
             $<$<CONFIG:Debug>:${BBB_FRAMEWORKS_FOUND_DEBUG}>
             $<$<CONFIG:Debug>:${BBB_SYSTEM_LIBS_DEBUG}>
             $<$<CONFIG:Debug>:BBB::comp1;BBB::comp2;BBB::compX> --> Not in this line
             APPEND)

Even though it is present further away in the file, just like all the libs mentionned in the list above (line 24):

########## COMPONENTS TARGET PROPERTIES Debug ########################################

    ########## COMPONENT BBB::culprit #############

        set(BBB_BBB_culprit_FRAMEWORKS_FOUND_DEBUG "")
        conan_find_apple_frameworks(BBB_BBB_culprit_FRAMEWORKS_FOUND_DEBUG "${BBB_BBB_culprit_FRAMEWORKS_DEBUG}" "${BBB_BBB_culprit_FRAMEWORK_DIRS_DEBUG}")

        set(BBB_BBB_culprit_LIBRARIES_TARGETS "")

...

I probably made a mistake at some point, but I don't understand how it manages to work when packaged with conan create but not as an editable one. If you have any idea, that would be fond of you to share it. Thanks in advance.

memsharded commented 1 year ago

Hi @Todiq

I think it was my mistake, I probably wasn't very clear, and it was an oversight from my side (I was responding issues while travelling):

The definition of components to be useful for editable mode, need to happen in the layout() method, not in the package_info() method. There isn't many references yet in the docs, we need to complete https://docs.conan.io/2/reference/conanfile/methods/layout.html#self-cpp: Components can be defined for self.cpp.source.components[...] and self.cpp.build.components[...] to specify their editable information (typically includedirs is self.cpp.source.components.. while libdirs is self.cpp.build.components.

Todiq commented 1 year ago

Hello @memsharded,

Here is my layout() now:

def layout(self):
    self.folders.source = "."
    self.folders.build = f"build/{self.settings.build_type}"
    self.folders.generators = f"{self.folders.build}/generators"

    for compname, comp in self._parsing_component_tree.items():
        conan_component = f"{self.name}_{compname.lower()}"
        if comp.is_lib is True:
            self.cpp.source.components[conan_component].includedirs = ["."]
            self.cpp.build.components[conan_component].libdirs = ["."]

Unfortunately, that still does not help. I must have made a mistake while following poco's recipe, but can't find where. As I mentionned in my previous message, the lib is not present in the list of components in build/Debug/generators/BBB-Target-debug.cmake:24, even if this line is displayed:

Conan: Component target declared 'BBB::culprit'

memsharded commented 1 year ago

Ok, I think at this point it would be good to setup a small repro case, with a package with a couple of components (dummy "hello world" libraries, with just 1 function), and a consumer, with a script that reproduce the issue. Do you think you could strip a copy of your project to create it, or would you like some help to set it up?

Todiq commented 1 year ago

Hey @memsharded,

You made an effort to answer quickly, but I could not follow up because it took me a while to create a reproducible case. I apologise for that. Anyway, I created a docker image containing it all: docker run -it --name todiq_test --rm todiq/conan All you have to do is execute conan build -verror beta/ --build="missing" to get the error.

The repository can be found here. Feel free to rebuild the image on your side if required.

While creating it, I noticed that, even though I never execute conan create (only editable mode), removing package_info() from package alpha raises an error. It makes me think that I should maybe use the self.cpp.package.components within my layout() method to hopefully fix the issue. Thank you again for your time.

memsharded commented 1 year ago

Hi @Todiq

Thanks very much for setting up the project. I managed to reproduce with the docker image. However, I have started to play with it with my own Linux docker, to understand better some things, and I am struggling with a couple of things:

It was failing to link shared libs, because of fPIC (maybe not an issue in Rocky Linux, but it was failing in my ubuntu)
The normal conan create is not working, so it is not only the editable mode, but the regular conan create doesn't work (I am adding a test_package to validate it, very recommended). This is expected because the CMakeLists.txt do not contain any installation code. I guess that you might have stripped that code, and the repro case covers exclusively the editable case.

While creating it, I noticed that, even though I never execute conan create (only editable mode), removing package_info() from package alpha raises an error. It makes me think that I should maybe use the self.cpp.package.components within my layout() method to hopefully fix the issue.

Yes, brilliant observation, I think that is probably the right approach.

I'll keep working on this later if possible, some quick hints:

Your configure() method is probably missing this:
```
    def configure(self):
```
if self.options.shared:
self.options.rm_safe("fPIC") self.options["zlib/*"].shared=True
```
Otherwise fPIC is disabled and it fails to link
```
Most likely, the package_info() can be simplified:
```
    def package_info(self):
```
self.cpp_info.set_property("cmake_file_name", self.name)
self.cpp_info.set_property("cmake_target_name", f"{self.name}::{self.name}")

        for compname, comp in self._alpha_component_tree.items():
                conan_component = f"{self.name}_{compname.lower()}"
                requires = [f"{self.name}_{dependency.lower()}" for dependency in comp.dependencies] + comp.external_dependencies

self.cpp_info.components[conan_component].set_property("cmake_target_name", f"{self.name}::{compname}")
self.cpp_info.components[conan_component].set_property("cmake_file_name", compname)
self.cpp_info.components[conan_component].names["cmake_find_package"] = compname

self.cpp_info.components[conan_component].names["cmake_find_package_multi"] = compname if comp.is_lib: self.cpp_info.components[conan_component].libs = [f"{compname}"] self.cpp_info.components[conan_component].requires = requires


First you don't need to support legacy 1.X generators like ``cmake_find_package``. Then, also, the default for ``cmake_target_name`` is already that, no need to define it. And overwriting the ``cmake_file_name`` to the ``compname`` is not correct, because it is just 1 file per package, not per component, so it has to be the name of the package, not the component.

I'd also strongly recommend using the test_package functionality, so a single conan create . is good to create and test the package quickly. I will try to submit some changes later with it.

Todiq commented 1 year ago

Hello @memsharded

Thanks a lot for your dedication. I reviewed your changes and merged them. Indeed, I removed a lot of things to stick with my editable issue, as I didn't want to pollute the sources (hence the removal of self.options.rm_safe("fPIC") for example). I removed many lines in my package_info() method, matching what you showed me, and it still works! (should I be surprised? :) ) However, editable mode still won't work for alpha. I tried replicating in layout() what you wrote in package_info(), but that would not fix the issue:

def layout(self):
    self.folders.source = "."
    self.folders.build = f"build/{self.settings.build_type}"
    self.folders.generators = f"{self.folders.build}/generators"

    for compname, comp in self._alpha_component_tree.items():
        if comp.is_lib:
            self.cpp.source.components[compname].includedirs = ["."] # I tried this line in and outside of the if condition
            self.cpp.build.components[compname].libdirs = ["."]

After setting alpha in editable mode, I ran:

conan build -verror beta/ --options "&:shared=True" --build=missing

and got:

-- Using Conan toolchain: /workspace/test_conan/Test/beta/build/Release/generators/conan_toolchain.cmake
-- Conan toolchain: Setting CMAKE_POSITION_INDEPENDENT_CODE=OFF (options.fPIC)
-- Conan toolchain: C++ Standard 17 with extensions ON
-- Conan toolchain: Setting BUILD_SHARED_LIBS = ON
-- Conan: Component target declared 'alpha::alpha'
-- Conan: Component target declared 'alpha::alpha1_1'
-- Conan: Component target declared 'alpha::alpha1_2'
-- Conan: Component target declared 'alpha::alpha2'
CMake Error at build/Release/generators/cmakedeps_macros.cmake:66 (message):
  Library 'alpha2' not found in package.  If 'alpha2' is a system library,
  declare it with 'cpp_info.system_libs' property
Call Stack (most recent call first):
  build/Release/generators/alpha-Target-release.cmake:24 (conan_package_library_targets)
  build/Release/generators/alphaTargets.cmake:26 (include)
  build/Release/generators/alpha-config.cmake:16 (include)
  CMakeLists.txt:5 (find_package)

-- Configuring incomplete, errors occurred!
ERROR: conanfile.py (beta/1.0): Error in build() method, line 50
        cmake.configure()
        ConanException: Error 1 while executing

Do you have some ideas? Thanks in advance

Todiq commented 1 year ago

Hello @memsharded

You must be very busy. Would it be possible to have a quick look when you have a minute please? Thank you very much for your help so far, I appreciate it.

Todiq commented 1 year ago

Hi @memsharded

Could you please give me a hand about this when you have a moment? Thanks in advance.

memsharded commented 1 year ago

Thanks @Todiq for resurfacing this. Indeed, we are extremely busy, it is very complicated to follow up, specially tickets that require investing more time.

I am doing some further checking (ongoing wip https://github.com/memsharded/test_conan/pull/new/review2), and I am seeing some unexpected behaviors, so I am going to reproduce in a test in Conan test suite first, to make sure things work fine.

memsharded commented 1 year ago

Submitting https://github.com/conan-io/conan/pull/13801, it seems editables + components work fine

Todiq commented 1 year ago

Hello @memsharded,

I merged your PR, thank you. However, after running the build.py, I still have the following errors. Did I miss something?

Log

```shell python3 build.py Reference 'alpha/1.0' in editable mode ======== Input profiles ======== Profile host: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=8 os=Linux [conf] tools.build:skip_test=True Profile build: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=8 os=Linux [conf] tools.build:skip_test=True ======== Computing dependency graph ======== Graph root conanfile.py (alpha/1.0): /home/carl/test_conan/Test/alpha/conanfile.py Requirements zlib/1.2.13#e377bee636333ae348d51ca90874e353 - Cache ======== Computing necessary packages ======== Requirements zlib/1.2.13#e377bee636333ae348d51ca90874e353:f940c1f7889f9018080dbeebdede5a67d6c3b30e#c17d6240d67d4f701aa6ee23db055ac4 - Cache ======== Installing packages ======== ======== Installing packages ======== zlib/1.2.13: Already installed! (1 of 1) WARN: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X: WARN: 'cpp_info.names' used in: zlib/1.2.13 ======== Finalizing install (deploy, generators) ======== conanfile.py (alpha/1.0): Calling generate() conanfile.py (alpha/1.0): Generators folder: /home/carl/test_conan/Test/alpha/build/Release/generators conanfile.py (alpha/1.0): CMakeToolchain generated: conan_toolchain.cmake conanfile.py (alpha/1.0): Preset 'conan-release' added to CMakePresets.json. Invoke it manually using 'cmake --preset conan-release' if using CMake>=3.23 conanfile.py (alpha/1.0): If your CMake version is not compatible with CMakePresets (<3.23) call cmake like: 'cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=/home/carl/test_conan/Test/alpha/build/Release/generators/conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Release' conanfile.py (alpha/1.0): CMakeToolchain generated: CMakePresets.json conanfile.py (alpha/1.0): CMakeToolchain generated: ../../../CMakeUserPresets.json conanfile.py (alpha/1.0): Generating aggregated env files conanfile.py (alpha/1.0): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] ======== Calling build() ======== conanfile.py (alpha/1.0): Calling build() conanfile.py (alpha/1.0): Running CMake.configure() conanfile.py (alpha/1.0): RUN: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="/home/carl/test_conan/Test/alpha/build/Release/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/carl/test_conan/Test/alpha" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DBUILD_TESTING="OFF" -DCMAKE_BUILD_TYPE="Release" "/home/carl/test_conan/Test/alpha/." -- Using Conan toolchain: /home/carl/test_conan/Test/alpha/build/Release/generators/conan_toolchain.cmake -- Conan toolchain: C++ Standard 17 with extensions ON -- Conan toolchain: Setting BUILD_SHARED_LIBS = ON -- The CXX compiler identification is GNU 8.5.0 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Conan: Target declared 'ZLIB::ZLIB' -- Configuring done (0.1s) -- Generating done (0.0s) CMake Warning: Manually-specified variables were not used by the project: BUILD_TESTING CMAKE_POLICY_DEFAULT_CMP0091 -- Build files have been written to: /home/carl/test_conan/Test/alpha/build/Release conanfile.py (alpha/1.0): Running CMake.build() conanfile.py (alpha/1.0): RUN: cmake --build "/home/carl/test_conan/Test/alpha/build/Release" -- -j8 [ 16%] Building CXX object alpha1/alpha1_1/CMakeFiles/alpha1_1.dir/src/alpha1_1.cpp.o [ 33%] Linking CXX shared library libalpha1_1.so [ 33%] Built target alpha1_1 [ 50%] Building CXX object alpha2/CMakeFiles/alpha2.dir/src/alpha2.cpp.o [ 66%] Building CXX object alpha1/alpha1_2/CMakeFiles/alpha1_2.dir/src/alpha1_2.cpp.o [ 83%] Linking CXX shared library libalpha2.so [ 83%] Built target alpha2 [100%] Linking CXX shared library libalpha1_2.so [100%] Built target alpha1_2 ======== Input profiles ======== Profile host: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=8 os=Linux [conf] tools.build:skip_test=True Profile build: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=8 os=Linux [conf] tools.build:skip_test=True ======== Computing dependency graph ======== Graph root conanfile.py (beta/1.0): /home/carl/test_conan/Test/beta/conanfile.py Requirements alpha/1.0 - Editable zlib/1.2.13#e377bee636333ae348d51ca90874e353 - Cache ======== Computing necessary packages ======== Requirements alpha/1.0:04902910cc170dfbfd8536063e51c0ee2e58fb32 - Editable zlib/1.2.13#e377bee636333ae348d51ca90874e353:f940c1f7889f9018080dbeebdede5a67d6c3b30e#c17d6240d67d4f701aa6ee23db055ac4 - Cache ======== Installing packages ======== ======== Installing packages ======== zlib/1.2.13: Already installed! (1 of 2) alpha/1.0: Rewriting files of editable package 'alpha' at '/home/carl/test_conan/Test/alpha/build/Release/generators' alpha/1.0: Calling generate() alpha/1.0: Generators folder: /home/carl/test_conan/Test/alpha/build/Release/generators alpha/1.0: CMakeToolchain generated: conan_toolchain.cmake alpha/1.0: CMakeToolchain generated: CMakePresets.json alpha/1.0: CMakeToolchain generated: ../../../CMakeUserPresets.json alpha/1.0: Generating aggregated env files alpha/1.0: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] WARN: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X: WARN: 'cpp_info.names' used in: zlib/1.2.13 ======== Finalizing install (deploy, generators) ======== conanfile.py (beta/1.0): Calling generate() conanfile.py (beta/1.0): Generators folder: /home/carl/test_conan/Test/beta/build/Release/generators conanfile.py (beta/1.0): CMakeToolchain generated: conan_toolchain.cmake conanfile.py (beta/1.0): Preset 'conan-release' added to CMakePresets.json. Invoke it manually using 'cmake --preset conan-release' if using CMake>=3.23 conanfile.py (beta/1.0): If your CMake version is not compatible with CMakePresets (<3.23) call cmake like: 'cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=/home/carl/test_conan/Test/beta/build/Release/generators/conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Release' conanfile.py (beta/1.0): CMakeToolchain generated: CMakePresets.json conanfile.py (beta/1.0): CMakeToolchain generated: ../../../CMakeUserPresets.json conanfile.py (beta/1.0): Generating aggregated env files conanfile.py (beta/1.0): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] ======== Calling build() ======== conanfile.py (beta/1.0): Calling build() conanfile.py (beta/1.0): Running CMake.configure() conanfile.py (beta/1.0): RUN: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="/home/carl/test_conan/Test/beta/build/Release/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/carl/test_conan/Test/beta" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DBUILD_TESTING="OFF" -DCMAKE_BUILD_TYPE="Release" "/home/carl/test_conan/Test/beta/." -- Using Conan toolchain: /home/carl/test_conan/Test/beta/build/Release/generators/conan_toolchain.cmake -- Conan toolchain: C++ Standard 17 with extensions ON -- The CXX compiler identification is GNU 8.5.0 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Conan: Component target declared 'alpha::alpha_alpha1_1' -- Conan: Component target declared 'alpha::alpha_alpha1_2' -- Conan: Component target declared 'alpha::alpha_alpha2' -- Conan: Target declared 'alpha::alpha' CMake Error at build/Release/generators/cmakedeps_macros.cmake:66 (message): Library 'alpha2' not found in package. If 'alpha2' is a system library, declare it with 'cpp_info.system_libs' property Call Stack (most recent call first): build/Release/generators/alpha-Target-release.cmake:24 (conan_package_library_targets) build/Release/generators/alphaTargets.cmake:26 (include) build/Release/generators/alpha-config.cmake:16 (include) CMakeLists.txt:5 (find_package) -- Configuring incomplete, errors occurred! ********************************************************* Recipe 'conanfile.py (beta/1.0)' cannot build its binary It is possible that this recipe is not Conan 2.0 ready If the recipe comes from ConanCenter check: https://conan.io/cci-v2.html If it is your recipe, check if it is updated to 2.0 ********************************************************* ERROR: conanfile.py (beta/1.0): Error in build() method, line 34 cmake.configure() ConanException: Error 1 while executing Traceback (most recent call last): File "/home/carl/test_conan/build.py", line 17, in run("conan build Test/beta") File "/home/carl/test_conan/build.py", line 6, in run raise Exception(f"Failed CMD: {cmd}") Exception: Failed CMD: conan build Test/beta ```

memsharded commented 1 year ago

I developed and tested it in Windows. It is very possible that some of the definitions of layouts are Windows-specific and fails in Linux system. Let me try to have a look and make it more portable.

memsharded commented 1 year ago

This https://github.com/Todiq/test_conan/pull/3 works both in WIndows and Linux.

The key would be also to simplify the CMakeLists.txt to achieve a binary output in CMake that is simple and homogenous across platforms, that would simplify the definition of components in the conanfile.py

memsharded commented 1 year ago

https://github.com/conan-io/conan/pull/13801 was merged, proving it works, and also https://github.com/Todiq/test_conan/pull/3 seems to finally fix this?

Todiq commented 1 year ago

Hello @memsharded,

Thank you for your attention. Your fix indeed works. I copied it to simplify my own recipe :

from conan import ConanFile
from conan.tools.cmake import CMake, CMakeToolchain, CMakeDeps, cmake_layout
from conan.tools.files import copy

required_conan_version = ">=2.0.0"

class Pkg(ConanFile):
    name = "castparsing"
    version = "1.0"

    # Binary configuration
    settings = "os", "compiler", "build_type", "arch"
    options = {
        "shared": [True, False],
        "fPIC": [True, False],
    }
    default_options = {
        "shared": False,
        "fPIC": True,
    }

    _parsing_components = {
        "a3sqlu": "common/a3sqlu",
        "memmng": "common/memmng",
        "mngtree": "common/mngtree",
        "noyau": "common/noyau",
        "syntree": "common/syntree",
        "tblsym": "common/tblsym",
        "lexel": "components/jee/lexel",
        "lexj11": "components/jee/lexj11",
        "lexjsp11": "components/jee/lexjsp11",
        "yacel": "components/jee/yacel",
        "yacj11": "components/jee/yacj11",
        "yacjsp11": "components/jee/yacjsp11",
        "lexbms": "components/mainframe/lexbms",
        "lexcics": "components/mainframe/lexcics",
        "lexcob": "components/mainframe/lexcob",
        "lexcsd": "components/mainframe/lexcsd",
        "lexims": "components/mainframe/lexims",
        "lexjcl": "components/mainframe/lexjcl",
        "sqltiny": "components/sql",
        "lexabap": "components/sap/lexabap",
        "lexcdsview": "components/sap/lexcdsview",
        "yacabap": "components/sap/yacabap"
    }

    def export_sources(self):
        copy(self, "*.c*", src=self.recipe_folder, dst=self.export_sources_folder)
        copy(self, "*.h*", src=self.recipe_folder, dst=self.export_sources_folder)
        copy(self, "*.inl", src=self.recipe_folder, dst=self.export_sources_folder)
        copy(self, "*.l", src=self.recipe_folder, dst=self.export_sources_folder)
        copy(self, "*CMakeLists.txt", src=self.recipe_folder, dst=self.export_sources_folder)

    def requirements(self):
        self.requires("castcore/1.0")
        self.requires("zlib/1.2.13")

    def config_options(self):
        if self.settings.os == "Windows":
            del self.options.fPIC

    def configure(self):
        if self.options.shared:
            self.options.rm_safe("fPIC")
        self.options["castcore/*"].shared=True
        self.options["zlib/*"].shared=True

    def layout(self):
        self.folders.source = "."
        self.folders.build = f"build/{self.settings.build_type}"
        self.folders.generators = f"{self.folders.build}/generators"
        bt_folder = f"/{self.settings.build_type}" if self.settings.compiler == "msvc" else ""

        for compname in self._parsing_components.items():
            component = compname[0]
            folder = compname[1]
            self.cpp.source.components[component].includedirs = [f"{folder}/include"]
            self.cpp.build.components[component].libdirs = [f"{folder}{bt_folder}"]
            self.cpp.build.components[component].bindirs = [f"{folder}{bt_folder}"]

    def generate(self):
        ct = CMakeToolchain(self)
        ct.generate()
        cd = CMakeDeps(self)
        cd.generate()

    def build(self):
        cmake = CMake(self)
        cmake.configure()
        cmake.build()

    def package(self):
        cmake = CMake(self)
        cmake.install()

    def package_info(self):
        for compname in self._parsing_components.items():
            component = compname[0]
            self.cpp_info.components[component].libs = [f"{component}"]

However, although it also fixed my initial error for my own case, I am now getting an include error (only in editable mode):

Log

```bash In file included from /home/carl/framework/FlowControl/src/FlowControlBlock.cpp:16: /home/carl/parsing/common/noyau/include/CastParsing/noyau/noyau.h:23:11: fatal error: CastParsing/tblsym/tbl_symb.h: No such file or directory #include ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. ```

The layout of the include files is virtually the same than the test repo.

I wonder if I made a mistake within my CMakeLists.txt for the lib called noyau. However, once again, the file looks almost identical with the ones in the repo. Is my layout() method wrong? Thanks in advance.

memsharded commented 1 year ago

I have been reading your recipe several times, but I don't see anything that could be wrong at plain sight. Sometimes depending on the compiler doing #include "...", instead of #include <...> solves some issues, but I don't think this is the case at the moment.

Maybe if you could try to replicate the error in the git project, that would really help.

Todiq commented 1 year ago

Hello @memsharded,

I have been trying the whole day, but I can neither reproduce it, nor can I fix it on my side.

I tried manually entering the dependencies within the layout() and package_info() methods (like you did) but that didn't help. Plus, looping through the deps that have been stored beforehand within the namedtuple works on the test repo (I commited).

It looks like the links of the libs in my castparsing package are ignored, thus leading to the include error. The dependencies are properly set in the configure() method and I made sure to link publicly in the CMakeLists.txt (with target_link_libraries. The only difference is that casparsing links with an external library of mine, called castcore, but I doubt it is the cause of the issue, since linking the latter with alpha_1_1 does not break the compilation for beta.

I can't be 100% sure, but at this point I must have a config error in my multiple CMakeLists.txt in my project, but comparing it side by side with the test repo didn't reveal any difference. Here is one example:

project(mngtree LANGUAGES CXX)

find_package(castcore)

add_library(${PROJECT_NAME} SHARED)
#add_library(castparsing::${PROJECT_NAME} ALIAS ${PROJECT_NAME})

target_sources(${PROJECT_NAME}
    PRIVATE
        src/mng_tree.cpp
        src/SignaturesSearchCtx.cpp
        src/stdafx.cpp
)

target_include_directories(${PROJECT_NAME}
    PUBLIC
        ${CMAKE_CURRENT_SOURCE_DIR}/include
    PRIVATE
        ${CMAKE_CURRENT_SOURCE_DIR}/src
)

target_link_libraries(${PROJECT_NAME}
    PUBLIC
        castcore::castcore
        memmng
        parsing
)

install(TARGETS ${PROJECT_NAME})

install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/include/CastParsing
        TYPE INCLUDE
)

NB: mngtree is the equivalent of alpha_1_2. It links within the same project with memmng (equivalent of alpha1_1) and with parsing (equivalent of myalpha, a lib interface).

If I link locally (in the test repo) beta with mngtree, the error I am getting indicates that it can't find the includes from castcore. Linking manually within beta's CMakeLists.txt fixes the error. However, it should automatically detect dependencies and link with them, just like it does between alphaand beta.

If you have any idea crossing your mind, I would gladly hear it. I am aware of the low quantity of information I am giving you. Not being able to reproduce is a nightmare. Thanks in advance.

memsharded commented 1 year ago

There is something that could probably be checked:

install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/include/CastParsing
        TYPE INCLUDE

This is typically an indicator that the targets are not fully defined at build time, otherwise the install(TARGETS is enough to also deploy/install the right headers to the right location. This could be the reason why it works for regular packages, but not for editables, because in editables, the install(... functionality is not called at all.

If you could share the generated xxx.cmake files for the failing consumer, the one that is missing the includes, that would help. Also the full log before the error, not just the error, could contain some hints.

Todiq commented 1 year ago

@memsharded

I understand what you are saying. However, I then don't get how it can work for the test repo, since I install the include dirs the same way and I can properly call the includes. Please note that although castparsing is in editable mode, castcore is not. To try everything out, I locally edited the beta package on the test repo to add anything castparsing related:

conanfile.py

```python from conan import ConanFile from conan.tools.cmake import CMake, CMakeToolchain, CMakeDeps, cmake_layout from conan.tools.files import copy, collect_libs required_conan_version = ">=2.0.0" class Pkg(ConanFile): name = "beta" version = "1.0" # Binary configuration settings = "os", "compiler", "build_type", "arch" def requirements(self): self.requires("alpha/1.0") self.requires("castparsing/1.0") def configure(self): self.options["alpha/*"].shared=True self.options["castparsing/*"].shared=True def layout(self): self.folders.source = "." self.folders.build = f"build/{self.settings.build_type}" self.folders.generators = f"{self.folders.build}/generators" def generate(self): ct = CMakeToolchain(self) ct.generate() cd = CMakeDeps(self) cd.generate() def build(self): cmake = CMake(self) cmake.configure() cmake.build() ```

CMakeLists.txt

```cmake set(PROJECT_NAME beta) cmake_minimum_required(VERSION 3.15) project(${PROJECT_NAME} LANGUAGES CXX) find_package(alpha) find_package(castparsing) add_executable(${PROJECT_NAME}) target_sources(${PROJECT_NAME} PRIVATE beta.cpp ) target_link_libraries(${PROJECT_NAME} PUBLIC alpha::alpha1_2 castparsing::mngtree #Equivalent to alpha_1_2 in castparsing ) install(TARGETS ${PROJECT_NAME}) ```

beta.cpp

```cpp #include #include int main() { alpha1_2(); return 0; } ```

If I run conan build test_conan/Test/beta, I get:

Log

```shell ======== Input profiles ======== Profile host: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=8 os=Linux [conf] tools.build:skip_test=True Profile build: [settings] arch=x86_64 build_type=Release compiler=gcc compiler.cppstd=gnu17 compiler.libcxx=libstdc++11 compiler.version=8 os=Linux [conf] tools.build:skip_test=True ======== Computing dependency graph ======== boost/1.81.0: running "/usr/bin/python3" -c "from __future__ import print_function; import sys; print('{}.{}'.format(sys.version_info[0], sys.version_info[1]))" boost/1.81.0: RUN: "/usr/bin/python3" -c "from __future__ import print_function; import sys; print('{}.{}'.format(sys.version_info[0], sys.version_info[1]))" Graph root conanfile.py (beta/1.0): /home/carl/test_conan/Test/beta/conanfile.py Requirements alpha/1.0 - Editable boost/1.81.0#9bd7ae86a881631b6cc76590621e470b - Cache bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d - Cache castcore/1.0#f70c45fa8a8a0eb3676fd83ddeefb963 - Cache castparsing/1.0 - Editable icu/72.1#8fa9fd1c317244e914cbd6f3fe403230 - Cache libbacktrace/cci.20210118#ec1aa63bbc10145c6a299e68e711670c - Cache libiconv/1.17#fa54397801cd96911a8294bc5fc76335 - Cache libxml2/2.10.3#1cbaf81341e07fcb0e61dadee882a193 - Cache pugixml/1.13#adee6d41b5ec42da7d4a39e0377dae34 - Cache xerces-c/3.2.3#fc0ec5173914f697013bb86fcb0f283c - Cache zlib/1.2.13#e377bee636333ae348d51ca90874e353 - Cache Test requirements gtest/1.13.0#b7b21d0d5503615d4ba0223f720b4c0b - Cache Build requirements b2/4.9.6#2e02945ad41739b8051247d14fe8297d - Cache ======== Computing necessary packages ======== Requirements alpha/1.0:04902910cc170dfbfd8536063e51c0ee2e58fb32 - Editable boost/1.81.0#9bd7ae86a881631b6cc76590621e470b:fe958fe55c7c08d3fd1a4c9e8b7ede53eb583486#c64fe56fde2115ae960636d2c9419fee - Cache bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d:75807e7575ecc0f6141344f7c4f5e86106b92ec2#7849b2509ab6d107caf31efcd5893386 - Skip castcore/1.0#f70c45fa8a8a0eb3676fd83ddeefb963:c2f81b6f8269108a79cdd975142d9b53b8d106e5#c5dd1cf5f2f02582b67e545ad41de717 - Cache castparsing/1.0:bbd16ac26c8544437c7a461b23107bba85f687e9 - Editable icu/72.1#8fa9fd1c317244e914cbd6f3fe403230:9a31ec3b2e0130e364172600173500c19b6120de#6f588b9b85f7d56ebff984bb413495d5 - Cache libbacktrace/cci.20210118#ec1aa63bbc10145c6a299e68e711670c:897e6fde718e72637d92b0b8ed929893a1976238#235a8ae8728834cacb95aa73c930c724 - Skip libiconv/1.17#fa54397801cd96911a8294bc5fc76335:897e6fde718e72637d92b0b8ed929893a1976238#515d0d49df85cfd007bdeaaffbed83e1 - Skip libxml2/2.10.3#1cbaf81341e07fcb0e61dadee882a193:20c3ad8b00f8bb5ab8149e5f62fe7341469ea46e#8d7e8551c413766a3e934e0f6caa24ea - Cache pugixml/1.13#adee6d41b5ec42da7d4a39e0377dae34:da39a3ee5e6b4b0d3255bfef95601890afd80709#ae530054ed7ac99330514c13a5bfde3a - Skip xerces-c/3.2.3#fc0ec5173914f697013bb86fcb0f283c:a812349d5cd48753a239b028fabd13cdb0b9e2cb#3e3a62e75867cfe12537d06c87f27b78 - Cache zlib/1.2.13#e377bee636333ae348d51ca90874e353:f940c1f7889f9018080dbeebdede5a67d6c3b30e#c17d6240d67d4f701aa6ee23db055ac4 - Cache Test requirements gtest/1.13.0#b7b21d0d5503615d4ba0223f720b4c0b:c045011a96715d1938d6d0006ead412758164ddc#f54d1554189b4b74377c027d2fa1412b - Skip Build requirements b2/4.9.6#2e02945ad41739b8051247d14fe8297d:63fead0844576fc02943e16909f08fcdddd6f44b#07e7cfb1a52c516a8bc2ff6109366e4e - Skip ======== Installing packages ======== ======== Installing packages ======== icu/72.1: Already installed! (1 of 8) zlib/1.2.13: Already installed! (2 of 8) boost/1.81.0: Already installed! (3 of 8) boost/1.81.0: running "/usr/bin/python3" -c "from __future__ import print_function; import sys; print('{}.{}'.format(sys.version_info[0], sys.version_info[1]))" boost/1.81.0: RUN: "/usr/bin/python3" -c "from __future__ import print_function; import sys; print('{}.{}'.format(sys.version_info[0], sys.version_info[1]))" boost/1.81.0: running "/usr/bin/python3" -c "from __future__ import print_function; import sys; print('{}.{}'.format(sys.version_info[0], sys.version_info[1]))" boost/1.81.0: RUN: "/usr/bin/python3" -c "from __future__ import print_function; import sys; print('{}.{}'.format(sys.version_info[0], sys.version_info[1]))" libxml2/2.10.3: Already installed! (4 of 8) libxml2/2.10.3: Appending PATH environment variable: /home/carl/.conan2/p/libxma1ffe0a72fda0/p/bin alpha/1.0: Rewriting files of editable package 'alpha' at '/home/carl/test_conan/Test/alpha/build/Release/generators' alpha/1.0: Calling generate() alpha/1.0: Generators folder: /home/carl/test_conan/Test/alpha/build/Release/generators alpha/1.0: CMakeToolchain generated: conan_toolchain.cmake alpha/1.0: CMakeToolchain generated: CMakePresets.json alpha/1.0: CMakeToolchain generated: ../../../CMakeUserPresets.json alpha/1.0: Generating aggregated env files alpha/1.0: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] xerces-c/3.2.3: Already installed! (6 of 8) castcore/1.0: Already installed! (7 of 8) castparsing/1.0: Rewriting files of editable package 'castparsing' at '/home/carl/parsing/build/Release/generators' castparsing/1.0: Calling generate() castparsing/1.0: Generators folder: /home/carl/parsing/build/Release/generators castparsing/1.0: CMakeToolchain generated: conan_toolchain.cmake castparsing/1.0: CMakeToolchain generated: CMakePresets.json castparsing/1.0: CMakeToolchain generated: ../../../CMakeUserPresets.json castparsing/1.0: Generating aggregated env files castparsing/1.0: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] WARN: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X: WARN: 'cpp_info.names' used in: icu/72.1, libxml2/2.10.3, zlib/1.2.13, boost/1.81.0 WARN: 'env_info' used in: icu/72.1, boost/1.81.0, libxml2/2.10.3 WARN: 'cpp_info.filenames' used in: libxml2/2.10.3, boost/1.81.0 WARN: 'user_info' used in: boost/1.81.0 WARN: 'cpp_info.build_modules' used in: libxml2/2.10.3 ======== Finalizing install (deploy, generators) ======== conanfile.py (beta/1.0): Calling generate() conanfile.py (beta/1.0): Generators folder: /home/carl/test_conan/Test/beta/build/Release/generators conanfile.py (beta/1.0): CMakeToolchain generated: conan_toolchain.cmake conanfile.py (beta/1.0): Preset 'conan-release' added to CMakePresets.json. Invoke it manually using 'cmake --preset conan-release' if using CMake>=3.23 conanfile.py (beta/1.0): If your CMake version is not compatible with CMakePresets (<3.23) call cmake like: 'cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=/home/carl/test_conan/Test/beta/build/Release/generators/conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Release' conanfile.py (beta/1.0): CMakeToolchain generated: CMakePresets.json conanfile.py (beta/1.0): CMakeToolchain generated: ../../../CMakeUserPresets.json conanfile.py (beta/1.0): Generating aggregated env files conanfile.py (beta/1.0): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] ======== Calling build() ======== conanfile.py (beta/1.0): Calling build() conanfile.py (beta/1.0): Running CMake.configure() conanfile.py (beta/1.0): RUN: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="/home/carl/test_conan/Test/beta/build/Release/generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/carl/test_conan/Test/beta" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DBUILD_TESTING="OFF" -DCMAKE_BUILD_TYPE="Release" "/home/carl/test_conan/Test/beta/." -- Using Conan toolchain: /home/carl/test_conan/Test/beta/build/Release/generators/conan_toolchain.cmake -- Conan toolchain: C++ Standard 17 with extensions ON -- The CXX compiler identification is GNU 8.5.0 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Conan: Component target declared 'alpha::myalpha' -- Conan: Component target declared 'alpha::alpha1_1' -- Conan: Component target declared 'alpha::alpha1_2' -- Conan: Component target declared 'alpha::alpha2' -- Conan: Target declared 'alpha::alpha' -- Conan: Target declared 'ZLIB::ZLIB' -- Conan: Component target declared 'castparsing::parsing' -- Conan: Component target declared 'castparsing::lexparse' -- Conan: Component target declared 'castparsing::memmng' -- Conan: Component target declared 'castparsing::mngtree' -- Conan: Target declared 'castparsing::castparsing' -- Conan: Component target declared 'castcore::castcore' -- Conan: Component target declared 'Boost::diagnostic_definitions' -- Conan: Component target declared 'Boost::disable_autolinking' -- Conan: Component target declared 'Boost::dynamic_linking' -- Conan: Component target declared 'Boost::headers' -- Conan: Component target declared 'Boost::boost' -- Conan: Component target declared 'boost::_libboost' -- Conan: Component target declared 'Boost::atomic' -- Conan: Component target declared 'Boost::container' -- Conan: Component target declared 'Boost::context' -- Conan: Component target declared 'Boost::date_time' -- Conan: Component target declared 'Boost::exception' -- Conan: Component target declared 'Boost::math' -- Conan: Component target declared 'Boost::math_c99' -- Conan: Component target declared 'Boost::math_c99f' -- Conan: Component target declared 'Boost::math_c99l' -- Conan: Component target declared 'Boost::math_tr1' -- Conan: Component target declared 'Boost::math_tr1f' -- Conan: Component target declared 'Boost::math_tr1l' -- Conan: Component target declared 'Boost::program_options' -- Conan: Component target declared 'Boost::python' -- Conan: Component target declared 'Boost::regex' -- Conan: Component target declared 'Boost::serialization' -- Conan: Component target declared 'Boost::stacktrace' -- Conan: Component target declared 'Boost::stacktrace_addr2line' -- Conan: Component target declared 'Boost::stacktrace_backtrace' -- Conan: Component target declared 'Boost::stacktrace_basic' -- Conan: Component target declared 'Boost::stacktrace_noop' -- Conan: Component target declared 'Boost::system' -- Conan: Component target declared 'Boost::test' -- Conan: Component target declared 'Boost::test_exec_monitor' -- Conan: Component target declared 'Boost::url' -- Conan: Component target declared 'Boost::wserialization' -- Conan: Component target declared 'boost::python39' -- Conan: Component target declared 'Boost::chrono' -- Conan: Component target declared 'Boost::coroutine' -- Conan: Component target declared 'Boost::filesystem' -- Conan: Component target declared 'Boost::json' -- Conan: Component target declared 'Boost::nowide' -- Conan: Component target declared 'Boost::numpy' -- Conan: Component target declared 'Boost::prg_exec_monitor' -- Conan: Component target declared 'Boost::random' -- Conan: Component target declared 'Boost::thread' -- Conan: Component target declared 'Boost::timer' -- Conan: Component target declared 'Boost::type_erasure' -- Conan: Component target declared 'Boost::unit_test_framework' -- Conan: Component target declared 'Boost::wave' -- Conan: Component target declared 'boost::numpy39' -- Conan: Component target declared 'Boost::contract' -- Conan: Component target declared 'Boost::fiber' -- Conan: Component target declared 'Boost::fiber_numa' -- Conan: Component target declared 'Boost::graph' -- Conan: Component target declared 'Boost::iostreams' -- Conan: Component target declared 'Boost::locale' -- Conan: Component target declared 'Boost::log' -- Conan: Component target declared 'Boost::log_setup' -- Conan: Target declared 'boost::boost' -- Conan: Target declared 'LibXml2::LibXml2' -- Conan: Including build module from '/home/carl/.conan2/p/libxma1ffe0a72fda0/p/lib/cmake/conan-official-libxml2-variables.cmake' -- Conan: Target declared 'XercesC::XercesC' -- Conan: Component target declared 'ICU::data' -- Conan: Component target declared 'ICU::dt' -- Conan: Component target declared 'ICU::uc' -- Conan: Component target declared 'ICU::i18n' -- Conan: Component target declared 'ICU::in' -- Conan: Component target declared 'ICU::io' -- Conan: Component target declared 'ICU::tu' -- Conan: Component target declared 'ICU::test' -- Conan: Target declared 'icu::icu' -- Configuring done (0.2s) -- Generating done (0.0s) CMake Warning: Manually-specified variables were not used by the project: BUILD_TESTING CMAKE_POLICY_DEFAULT_CMP0091 -- Build files have been written to: /home/carl/test_conan/Test/beta/build/Release conanfile.py (beta/1.0): Running CMake.build() conanfile.py (beta/1.0): RUN: cmake --build "/home/carl/test_conan/Test/beta/build/Release" -- -j8 [ 50%] Building CXX object CMakeFiles/beta.dir/beta.cpp.o In file included from /home/carl/test_conan/Test/beta/beta.cpp:2: /home/carl/parsing/common/mngtree/include/CastParsing/mngtree/yac_macr.h:10:10: fatal error: CAST/Fundamental/core/ICollator.h: No such file or directory #include ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [CMakeFiles/beta.dir/build.make:76: CMakeFiles/beta.dir/beta.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/beta.dir/all] Error 2 gmake: *** [Makefile:136: all] Error 2 ********************************************************* Recipe 'conanfile.py (beta/1.0)' cannot build its binary It is possible that this recipe is not Conan 2.0 ready If the recipe comes from ConanCenter check: https://conan.io/cci-v2.html If it is your recipe, check if it is updated to 2.0 ********************************************************* ERROR: conanfile.py (beta/1.0): Error in build() method, line 37 cmake.build() ConanException: Error 2 while executing ```

We can see that an include from castcore is not detected. Now, here are its generated .cmake files, taken from test_conan/Test/beta/build/Release/generators:

castcore-Target-release.cmake

```cmake # Avoid multiple calls to find_package to append duplicated properties to the targets include_guard()########### VARIABLES ####################################################################### ############################################################################################# set(castcore_FRAMEWORKS_FOUND_RELEASE "") # Will be filled later conan_find_apple_frameworks(castcore_FRAMEWORKS_FOUND_RELEASE "${castcore_FRAMEWORKS_RELEASE}" "${castcore_FRAMEWORK_DIRS_RELEASE}") set(castcore_LIBRARIES_TARGETS "") # Will be filled later ######## Create an interface target to contain all the dependencies (frameworks, system and conan deps) if(NOT TARGET castcore_DEPS_TARGET) add_library(castcore_DEPS_TARGET INTERFACE IMPORTED) endif() set_property(TARGET castcore_DEPS_TARGET PROPERTY INTERFACE_LINK_LIBRARIES $<$:${castcore_FRAMEWORKS_FOUND_RELEASE}> $<$:${castcore_SYSTEM_LIBS_RELEASE}> $<$:boost::boost;LibXml2::LibXml2;XercesC::XercesC;icu::icu> APPEND) ####### Find the libraries declared in cpp_info.libs, create an IMPORTED target for each one and link the ####### castcore_DEPS_TARGET to all of them conan_package_library_targets("${castcore_LIBS_RELEASE}" # libraries "${castcore_LIB_DIRS_RELEASE}" # package_libdir "${castcore_BIN_DIRS_RELEASE}" # package_bindir "${castcore_LIBRARY_TYPE_RELEASE}" "${castcore_IS_HOST_WINDOWS_RELEASE}" castcore_DEPS_TARGET castcore_LIBRARIES_TARGETS # out_libraries_targets "_RELEASE" "castcore" # package_name "${castcore_NO_SONAME_MODE_RELEASE}") # soname # FIXME: What is the result of this for multi-config? All configs adding themselves to path? set(CMAKE_MODULE_PATH ${castcore_BUILD_DIRS_RELEASE} ${CMAKE_MODULE_PATH}) ########## COMPONENTS TARGET PROPERTIES Release ######################################## ########## COMPONENT castcore::castcore ############# set(castcore_castcore_castcore_FRAMEWORKS_FOUND_RELEASE "") conan_find_apple_frameworks(castcore_castcore_castcore_FRAMEWORKS_FOUND_RELEASE "${castcore_castcore_castcore_FRAMEWORKS_RELEASE}" "${castcore_castcore_castcore_FRAMEWORK_DIRS_RELEASE}") set(castcore_castcore_castcore_LIBRARIES_TARGETS "") ######## Create an interface target to contain all the dependencies (frameworks, system and conan deps) if(NOT TARGET castcore_castcore_castcore_DEPS_TARGET) add_library(castcore_castcore_castcore_DEPS_TARGET INTERFACE IMPORTED) endif() set_property(TARGET castcore_castcore_castcore_DEPS_TARGET PROPERTY INTERFACE_LINK_LIBRARIES $<$:${castcore_castcore_castcore_FRAMEWORKS_FOUND_RELEASE}> $<$:${castcore_castcore_castcore_SYSTEM_LIBS_RELEASE}> $<$:${castcore_castcore_castcore_DEPENDENCIES_RELEASE}> APPEND) ####### Find the libraries declared in cpp_info.component["xxx"].libs, ####### create an IMPORTED target for each one and link the 'castcore_castcore_castcore_DEPS_TARGET' to all of them conan_package_library_targets("${castcore_castcore_castcore_LIBS_RELEASE}" "${castcore_castcore_castcore_LIB_DIRS_RELEASE}" "${castcore_castcore_castcore_BIN_DIRS_RELEASE}" # package_bindir "${castcore_castcore_castcore_LIBRARY_TYPE_RELEASE}" "${castcore_castcore_castcore_IS_HOST_WINDOWS_RELEASE}" castcore_castcore_castcore_DEPS_TARGET castcore_castcore_castcore_LIBRARIES_TARGETS "_RELEASE" "castcore_castcore_castcore" "${castcore_castcore_castcore_NO_SONAME_MODE_RELEASE}") ########## TARGET PROPERTIES ##################################### set_property(TARGET castcore::castcore PROPERTY INTERFACE_LINK_LIBRARIES $<$:${castcore_castcore_castcore_OBJECTS_RELEASE}> $<$:${castcore_castcore_castcore_LIBRARIES_TARGETS}> APPEND) if("${castcore_castcore_castcore_LIBS_RELEASE}" STREQUAL "") # If the component is not declaring any "cpp_info.components['foo'].libs" the system, frameworks etc are not # linked to the imported targets and we need to do it to the global target set_property(TARGET castcore::castcore PROPERTY INTERFACE_LINK_LIBRARIES castcore_castcore_castcore_DEPS_TARGET APPEND) endif() set_property(TARGET castcore::castcore PROPERTY INTERFACE_LINK_OPTIONS $<$:${castcore_castcore_castcore_LINKER_FLAGS_RELEASE}> APPEND) set_property(TARGET castcore::castcore PROPERTY INTERFACE_INCLUDE_DIRECTORIES $<$:${castcore_castcore_castcore_INCLUDE_DIRS_RELEASE}> APPEND) set_property(TARGET castcore::castcore PROPERTY INTERFACE_LINK_DIRECTORIES $<$:${castcore_castcore_castcore_LIB_DIRS_RELEASE}> APPEND) set_property(TARGET castcore::castcore PROPERTY INTERFACE_COMPILE_DEFINITIONS $<$:${castcore_castcore_castcore_COMPILE_DEFINITIONS_RELEASE}> APPEND) set_property(TARGET castcore::castcore PROPERTY INTERFACE_COMPILE_OPTIONS $<$:${castcore_castcore_castcore_COMPILE_OPTIONS_RELEASE}> APPEND) ########## AGGREGATED GLOBAL TARGET WITH THE COMPONENTS ##################### set_property(TARGET castcore::castcore PROPERTY INTERFACE_LINK_LIBRARIES castcore::castcore APPEND) ########## For the modules (FindXXX) set(castcore_LIBRARIES_RELEASE castcore::castcore) ```

castcore-config-version.cmake

```cmake set(PACKAGE_VERSION "1.0") if(PACKAGE_VERSION VERSION_LESS PACKAGE_FIND_VERSION) set(PACKAGE_VERSION_COMPATIBLE FALSE) else() if("1.0" MATCHES "^([0-9]+)\\.") set(CVF_VERSION_MAJOR ${CMAKE_MATCH_1}) else() set(CVF_VERSION_MAJOR "1.0") endif() if(PACKAGE_FIND_VERSION_MAJOR STREQUAL CVF_VERSION_MAJOR) set(PACKAGE_VERSION_COMPATIBLE TRUE) else() set(PACKAGE_VERSION_COMPATIBLE FALSE) endif() if(PACKAGE_FIND_VERSION STREQUAL PACKAGE_VERSION) set(PACKAGE_VERSION_EXACT TRUE) endif() endif() ```

castcore-config.cmake

```cmake ########## MACROS ########################################################################### ############################################################################################# # Requires CMake > 3.15 if(${CMAKE_VERSION} VERSION_LESS "3.15") message(FATAL_ERROR "The 'CMakeDeps' generator only works with CMake >= 3.15") endif() if(castcore_FIND_QUIETLY) set(castcore_MESSAGE_MODE VERBOSE) else() set(castcore_MESSAGE_MODE STATUS) endif() include(${CMAKE_CURRENT_LIST_DIR}/cmakedeps_macros.cmake) include(${CMAKE_CURRENT_LIST_DIR}/castcoreTargets.cmake) include(CMakeFindDependencyMacro) check_build_type_defined() foreach(_DEPENDENCY ${castcore_FIND_DEPENDENCY_NAMES} ) # Check that we have not already called a find_package with the transitive dependency if(NOT ${_DEPENDENCY}_FOUND) find_dependency(${_DEPENDENCY} REQUIRED ${${_DEPENDENCY}_FIND_MODE}) endif() endforeach() set(castcore_VERSION_STRING "1.0") set(castcore_INCLUDE_DIRS ${castcore_INCLUDE_DIRS_RELEASE} ) set(castcore_INCLUDE_DIR ${castcore_INCLUDE_DIRS_RELEASE} ) set(castcore_LIBRARIES ${castcore_LIBRARIES_RELEASE} ) set(castcore_DEFINITIONS ${castcore_DEFINITIONS_RELEASE} ) # Only the first installed configuration is included to avoid the collision foreach(_BUILD_MODULE ${castcore_BUILD_MODULES_PATHS_RELEASE} ) message(${castcore_MESSAGE_MODE} "Conan: Including build module from '${_BUILD_MODULE}'") include(${_BUILD_MODULE}) endforeach() ```

beta castcore-release-x86_64-data.cmake

```cmake ########### AGGREGATED COMPONENTS AND DEPENDENCIES FOR THE MULTI CONFIG ##################### ############################################################################################# list(APPEND castcore_COMPONENT_NAMES castcore::castcore) list(REMOVE_DUPLICATES castcore_COMPONENT_NAMES) list(APPEND castcore_FIND_DEPENDENCY_NAMES Boost libxml2 XercesC ICU) list(REMOVE_DUPLICATES castcore_FIND_DEPENDENCY_NAMES) set(Boost_FIND_MODE "NO_MODULE") set(libxml2_FIND_MODE "NO_MODULE") set(XercesC_FIND_MODE "NO_MODULE") set(ICU_FIND_MODE "NO_MODULE") ########### VARIABLES ####################################################################### ############################################################################################# set(castcore_PACKAGE_FOLDER_RELEASE "/home/carl/.conan2/p/castcbb80980f499b3/p") set(castcore_BUILD_MODULES_PATHS_RELEASE ) set(castcore_INCLUDE_DIRS_RELEASE ) set(castcore_RES_DIRS_RELEASE ) set(castcore_DEFINITIONS_RELEASE ) set(castcore_SHARED_LINK_FLAGS_RELEASE ) set(castcore_EXE_LINK_FLAGS_RELEASE ) set(castcore_OBJECTS_RELEASE ) set(castcore_COMPILE_DEFINITIONS_RELEASE ) set(castcore_COMPILE_OPTIONS_C_RELEASE ) set(castcore_COMPILE_OPTIONS_CXX_RELEASE ) set(castcore_LIB_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/lib") set(castcore_BIN_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/bin") set(castcore_LIBRARY_TYPE_RELEASE SHARED) set(castcore_IS_HOST_WINDOWS_RELEASE 0) set(castcore_LIBS_RELEASE ) set(castcore_SYSTEM_LIBS_RELEASE ) set(castcore_FRAMEWORK_DIRS_RELEASE ) set(castcore_FRAMEWORKS_RELEASE ) set(castcore_BUILD_DIRS_RELEASE ) set(castcore_NO_SONAME_MODE_RELEASE FALSE) # COMPOUND VARIABLES set(castcore_COMPILE_OPTIONS_RELEASE "$<$:${castcore_COMPILE_OPTIONS_CXX_RELEASE}>" "$<$:${castcore_COMPILE_OPTIONS_C_RELEASE}>") set(castcore_LINKER_FLAGS_RELEASE "$<$,SHARED_LIBRARY>:${castcore_SHARED_LINK_FLAGS_RELEASE}>" "$<$,MODULE_LIBRARY>:${castcore_SHARED_LINK_FLAGS_RELEASE}>" "$<$,EXECUTABLE>:${castcore_EXE_LINK_FLAGS_RELEASE}>") set(castcore_COMPONENTS_RELEASE castcore::castcore) ########### COMPONENT castcore::castcore VARIABLES ############################################ set(castcore_castcore_castcore_INCLUDE_DIRS_RELEASE ) set(castcore_castcore_castcore_LIB_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/lib") set(castcore_castcore_castcore_BIN_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/bin") set(castcore_castcore_castcore_LIBRARY_TYPE_RELEASE SHARED) set(castcore_castcore_castcore_IS_HOST_WINDOWS_RELEASE 0) set(castcore_castcore_castcore_RES_DIRS_RELEASE ) set(castcore_castcore_castcore_DEFINITIONS_RELEASE ) set(castcore_castcore_castcore_OBJECTS_RELEASE ) set(castcore_castcore_castcore_COMPILE_DEFINITIONS_RELEASE ) set(castcore_castcore_castcore_COMPILE_OPTIONS_C_RELEASE "") set(castcore_castcore_castcore_COMPILE_OPTIONS_CXX_RELEASE "") set(castcore_castcore_castcore_LIBS_RELEASE ) set(castcore_castcore_castcore_SYSTEM_LIBS_RELEASE ) set(castcore_castcore_castcore_FRAMEWORK_DIRS_RELEASE ) set(castcore_castcore_castcore_FRAMEWORKS_RELEASE ) set(castcore_castcore_castcore_DEPENDENCIES_RELEASE ) set(castcore_castcore_castcore_SHARED_LINK_FLAGS_RELEASE ) set(castcore_castcore_castcore_EXE_LINK_FLAGS_RELEASE ) set(castcore_castcore_castcore_NO_SONAME_MODE_RELEASE FALSE) # COMPOUND VARIABLES set(castcore_castcore_castcore_LINKER_FLAGS_RELEASE $<$,SHARED_LIBRARY>:${castcore_castcore_castcore_SHARED_LINK_FLAGS_RELEASE}> $<$,MODULE_LIBRARY>:${castcore_castcore_castcore_SHARED_LINK_FLAGS_RELEASE}> $<$,EXECUTABLE>:${castcore_castcore_castcore_EXE_LINK_FLAGS_RELEASE}> ) set(castcore_castcore_castcore_COMPILE_OPTIONS_RELEASE "$<$:${castcore_castcore_castcore_COMPILE_OPTIONS_CXX_RELEASE}>" "$<$:${castcore_castcore_castcore_COMPILE_OPTIONS_C_RELEASE}>") ```

We can indeed see that castcore_castcore_castcore_INCLUDE_DIRS_RELEASE & castcore_castcore_castcore_LIBS_RELEASE are both empty, whereas, in the castcore-release-x86_64-data.cmake found in the parsing/build/Release/generators dir, they are not:

parsing castcore-release-x86_64-data.cmake

```cmake ########### AGGREGATED COMPONENTS AND DEPENDENCIES FOR THE MULTI CONFIG ##################### ############################################################################################# list(APPEND castcore_COMPONENT_NAMES castcore::castcore) list(REMOVE_DUPLICATES castcore_COMPONENT_NAMES) list(APPEND castcore_FIND_DEPENDENCY_NAMES Boost libxml2 XercesC ICU) list(REMOVE_DUPLICATES castcore_FIND_DEPENDENCY_NAMES) set(Boost_FIND_MODE "NO_MODULE") set(libxml2_FIND_MODE "NO_MODULE") set(XercesC_FIND_MODE "NO_MODULE") set(ICU_FIND_MODE "NO_MODULE") ########### VARIABLES ####################################################################### ############################################################################################# set(castcore_PACKAGE_FOLDER_RELEASE "/home/carl/.conan2/p/castcbb80980f499b3/p") set(castcore_BUILD_MODULES_PATHS_RELEASE ) set(castcore_INCLUDE_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/include") set(castcore_RES_DIRS_RELEASE ) set(castcore_DEFINITIONS_RELEASE ) set(castcore_SHARED_LINK_FLAGS_RELEASE ) set(castcore_EXE_LINK_FLAGS_RELEASE ) set(castcore_OBJECTS_RELEASE ) set(castcore_COMPILE_DEFINITIONS_RELEASE ) set(castcore_COMPILE_OPTIONS_C_RELEASE ) set(castcore_COMPILE_OPTIONS_CXX_RELEASE ) set(castcore_LIB_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/lib") set(castcore_BIN_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/bin") set(castcore_LIBRARY_TYPE_RELEASE SHARED) set(castcore_IS_HOST_WINDOWS_RELEASE 0) set(castcore_LIBS_RELEASE castcore) set(castcore_SYSTEM_LIBS_RELEASE ) set(castcore_FRAMEWORK_DIRS_RELEASE ) set(castcore_FRAMEWORKS_RELEASE ) set(castcore_BUILD_DIRS_RELEASE ) set(castcore_NO_SONAME_MODE_RELEASE FALSE) # COMPOUND VARIABLES set(castcore_COMPILE_OPTIONS_RELEASE "$<$:${castcore_COMPILE_OPTIONS_CXX_RELEASE}>" "$<$:${castcore_COMPILE_OPTIONS_C_RELEASE}>") set(castcore_LINKER_FLAGS_RELEASE "$<$,SHARED_LIBRARY>:${castcore_SHARED_LINK_FLAGS_RELEASE}>" "$<$,MODULE_LIBRARY>:${castcore_SHARED_LINK_FLAGS_RELEASE}>" "$<$,EXECUTABLE>:${castcore_EXE_LINK_FLAGS_RELEASE}>") set(castcore_COMPONENTS_RELEASE castcore::castcore) ########### COMPONENT castcore::castcore VARIABLES ############################################ set(castcore_castcore_castcore_INCLUDE_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/include") set(castcore_castcore_castcore_LIB_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/lib") set(castcore_castcore_castcore_BIN_DIRS_RELEASE "${castcore_PACKAGE_FOLDER_RELEASE}/bin") set(castcore_castcore_castcore_LIBRARY_TYPE_RELEASE SHARED) set(castcore_castcore_castcore_IS_HOST_WINDOWS_RELEASE 0) set(castcore_castcore_castcore_RES_DIRS_RELEASE ) set(castcore_castcore_castcore_DEFINITIONS_RELEASE ) set(castcore_castcore_castcore_OBJECTS_RELEASE ) set(castcore_castcore_castcore_COMPILE_DEFINITIONS_RELEASE ) set(castcore_castcore_castcore_COMPILE_OPTIONS_C_RELEASE "") set(castcore_castcore_castcore_COMPILE_OPTIONS_CXX_RELEASE "") set(castcore_castcore_castcore_LIBS_RELEASE castcore) set(castcore_castcore_castcore_SYSTEM_LIBS_RELEASE ) set(castcore_castcore_castcore_FRAMEWORK_DIRS_RELEASE ) set(castcore_castcore_castcore_FRAMEWORKS_RELEASE ) set(castcore_castcore_castcore_DEPENDENCIES_RELEASE ) set(castcore_castcore_castcore_SHARED_LINK_FLAGS_RELEASE ) set(castcore_castcore_castcore_EXE_LINK_FLAGS_RELEASE ) set(castcore_castcore_castcore_NO_SONAME_MODE_RELEASE FALSE) # COMPOUND VARIABLES set(castcore_castcore_castcore_LINKER_FLAGS_RELEASE $<$,SHARED_LIBRARY>:${castcore_castcore_castcore_SHARED_LINK_FLAGS_RELEASE}> $<$,MODULE_LIBRARY>:${castcore_castcore_castcore_SHARED_LINK_FLAGS_RELEASE}> $<$,EXECUTABLE>:${castcore_castcore_castcore_EXE_LINK_FLAGS_RELEASE}> ) set(castcore_castcore_castcore_COMPILE_OPTIONS_RELEASE "$<$:${castcore_castcore_castcore_COMPILE_OPTIONS_CXX_RELEASE}>" "$<$:${castcore_castcore_castcore_COMPILE_OPTIONS_C_RELEASE}>") ```

Here is the conanfile for castcore if that can help:

conanfile.py

```python from conan import ConanFile from conan.tools.cmake import CMake, CMakeToolchain, CMakeDeps, cmake_layout from conan.tools.files import copy, collect_libs required_conan_version = ">=2.0.0" class Pkg(ConanFile): name = "castcore" version = "1.0" # Binary configuration settings = "os", "compiler", "build_type", "arch" options = { "shared": [True, False], "fPIC": [True, False], } default_options = { "shared": False, "fPIC": True, } def export_sources(self): copy(self, "*.c*", src=self.recipe_folder, dst=self.export_sources_folder) copy(self, "*.h*", src=self.recipe_folder, dst=self.export_sources_folder) copy(self, "*CMakeLists.txt", src=self.recipe_folder, dst=self.export_sources_folder) copy(self, "*.inl", src=self.recipe_folder, dst=self.export_sources_folder) def build_requirements(self): self.test_requires("gtest/1.13.0") def requirements(self): if self.settings.os == "Windows": self.requires("libxslt/1.1.34") self.requires("icu/72.1") self.requires("boost/1.81.0") self.requires("libxml2/2.10.3") self.requires("pugixml/1.13") self.requires("xerces-c/3.2.3") def config_options(self): if self.settings.os == "Windows": del self.options.fPIC def configure(self): if self.options.shared: self.options.rm_safe("fPIC") if self.settings.os == "Windows": self.options["libxslt/*"].shared=True else: self.options["icu/*"].data_packaging="library" self.options["pugixml/*"].header_only=True self.options["icu/*"].shared=True self.options["boost/*"].shared=True self.options["boost/*"].without_python=False self.options["boost/*"].zlib=False self.options["libxml2/*"].shared=True self.options["libxml2/*"].zlib=False self.options["xerces-c/*"].shared=True self.options["xerces-c/*"].char_type="char16_t" self.options["xerces-c/*"].transcoder="icu" def layout(self): self.folders.source = "." self.folders.build = f"build/{self.settings.build_type}" self.folders.generators = f"{self.folders.build}/generators" self.cpp.source.includedirs = ["."] # self.cpp.build.libdirs = ["."] # self.cpp.build.bindirs = ["."] self.cpp.build.libdirs = ["CAST/Fundamental/core"] self.cpp.build.bindirs = ["CAST/Fundamental/core"] def generate(self): ct = CMakeToolchain(self, generator="Ninja") ct.generate() cd = CMakeDeps(self) cd.generate() def build(self): cmake = CMake(self) cmake.configure() cmake.build() def package(self): copy(self, "*.h*", src=self.source_folder, dst=f"{self.package_folder}/include") copy(self, "*.inl", src=self.source_folder, dst=f"{self.package_folder}/include") copy(self, "*.so", src=self.build_folder, dst=f"{self.package_folder}/lib", keep_path=False) copy(self, "*.lib", src=self.build_folder, dst=f"{self.package_folder}/lib", keep_path=False) copy(self, "*.dylib", src=self.build_folder, dst=f"{self.package_folder}/lib", keep_path=False) copy(self, "*.dll", src=self.build_folder, dst=f"{self.package_folder}/bin", keep_path=False) def package_info(self): # self.cpp_info.libdirs = ["lib", "lib64"] # self.cpp_info.libs = collect_libs(self) self.cpp_info.components["castcore"].libs = [f"castcore"] ```

Hope that helps. Thanks!

Todiq commented 1 year ago

Hello @memsharded,

I supposed you were wrapping up release 1.60. Do you have more free time now to have a look at this? Thanks in advance.

Todiq commented 1 year ago

Hello @memsharded,

Do you happen to have some time to look at this by any chance? Thanks in advance.

Todiq commented 9 months ago

Finally figured it myself. Had to configure the layout method to better match my need. Thank you!

conan-io / conan

[question] Conan editables with multilibs packages #13639

What is your question?

Have you read the CONTRIBUTING guide?