[question] conan[1.60] editable how work?

[mangeMelon]

What is your question?

Hello everyone, Intermediate beginner on Conan, please let me know if any information is missing.

conan --version Conan version 1.60.1 (installed by pip , sans venv) python --version Python 3.10.12

(windows wsl) ubuntu 22.04 and debian bulleye physics

as part of a project with several modules we used conan for managing dependencies.

I understand from my reading of issue that the layout.ini file is no longer used.

I guess my problem comes from my conanfile.py file, I have the bare minimum to generate a package and consume it.

I manage to follow the examples, it seems simple but I can't transpose.

My conanfile file does not contain the basic layout() method, if I add it the path of my toolchain is modified if I use exports_sources = "CMakeLists.txt", "src/*", "include/*" then the conan cache does contain the copies the source() method fails because the directory is not empty so can't git clone

Thanks for the documentation and examples https://docs.conan.io/1/developing_packages/editable_packages.html#editable-packages https://docs.conan.io/1/developing_packages/package_layout.html#package-layout https://docs.conan.io/1/reference/commands/development/editable.html?highlight=editable https://docs.conan.io/1/reference/conanfile/attributes.html#cpp-info-attributes-reference https://docs.conan.io/1/reference/conanfile/methods.html#layout https://docs.conan.io/1/reference/conanfile/tools/cmake/cmake_layout.html https://docs.conan.io/1/reference/conanfile/tools/cmake/cmake_layout.html#cmake-layout https://docs.conan.io/1/reference/conanfile/tools/layout.html?highlight=layout https://docs.conan.io/1/developing_packages/editable_packages.html#editable-packages https://github.com/conan-io/examples https://github.com/conan-io/examples/blob/master/features/editable

https://docs.conan.io/2/examples/conanfile/layout.html?highlight=layout https://docs.conan.io/2/examples/conanfile/layout/third_party_libraries.html https://docs.conan.io/2/reference/conanfile/methods/layout.html#reference-conanfile-methods-layout https://docs.conan.io/2/tutorial/developing_packages/editable_packages.html https://docs.conan.io/2/tutorial/developing_packages/package_layout.html#developing-packages-layout https://docs.conan.io/2/tutorial/developing_packages/package_layout.html#tutorial-package-layout https://github.com/conan-io/examples2/tree/main/examples/conanfile/layout

Here is my situation: I have three modules so three packages

Package A(lib) -consume-> Package B(lib) -consume-> Package C(bin)

I make corrections in Package A and verify that they are effective in Package C

I added Packages A and B in editable mode as I understand the examples if I build A then B; package C consumes editable versions

My PROBLEM is that the packages are empty, I can't generate version of package B, library A is missing while package A was successfully generated, I find library A in my repository

the Conan editable list command returns me the packages A and B with the paths to my git repositories

the install command told me that package A was "editable" and not "Cache" or "Download"

My workaround is by creating WIP packets, but that's too cumbersome for each change

Module tree Module_name |FILE_CMakeLists.txt |FILE_conanfile.py |DIR_bin | |DIR_OS(linux ou windows) | | |DIR_Release | | |DIR_Debug |DIR_src(cpp) |DIR_inc(header) |DIR_lib(lib_externe) |DIR_test | |DIR_unitaire | |DIR_fonctionnel |DIR_obj(build) | |DIR_OS(linux ou windows) | | |DIR_Release | | |DIR_Debug |DIR_conan_build(generator) | |DIR_OS(linux ou windows) | | |DIR_Release | | |DIR_Debug

Also is it possible to create a Conan package with just lib.a and and header (external supply)

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @mangeMelon

If you are a beginner with Conan, please do not use Conan 1.X and start directly with Conan 2.0.

the right way to progress to learn up to editables is to follow the tutorial (step by step, from the beginning) in: https://docs.conan.io/2/tutorial until you reach the editables section in https://docs.conan.io/2/tutorial/developing_packages/editable_packages.html

That section even contain full working examples in the examples2 project, that can be very helpful to have something running to understand how it works.

[memsharded]

Hi @mangeMelon

Did you manage to check that? In Conan 2.0 the --build=editable can instruct to automatically build the editable dependencies. In Conan 1.X this doesn't exist, and it is necessary to do conan build . before installing the consumers.

[mangeMelon]

Hello @memsharded,

So we started using Conan with version 1.59 (latest), Conan 2 was in beta.

When it was released, we looked and saw that several commands had logic changes, and the migration documentation was not finalized. And as we built our workflow on 1.59, we didn't want to break things by migrating to V2.

recently On Reddit, I read that there were incompatibilities between packages generated on one version or the other.

I tested the examples of v1 and v2, and I can follow them without problem. The examples are simple, but if I apply the same logic to our packages it doesn't work.

"In Conan 2.0 the --build=editable" So no, I didn't know about it, after research you must refer to this part https://docs.conan.io/2/tutorial/developing_packages/editable_packages.html#building-editable-dependencies

This meets my need for multiple editable packages, but premature, because I can't make a package editable.

I am attaching my conanfile conanfile_ori.py.txt copy taking into account the examples conanfile_edit.py.txt

I suppose I need to review the layout() function but its contents elude me

[memsharded]

Hi @mangeMelon

Thanks for your feedback. I have been having a look to your recipes, but I am failing to understand what could be happening. The edit recipe is using some internal source code, so I can't reproduce that either.

One of the main ideas when using editables is that you do that because you have the conanfile.py together with the source code that you are packaging. In that sense is not usual to have a git.clone in the source() method of that recipe, because the recipe is already in the source repo.

It is still possible to have an editable recipe for third-party code, where the source code is in a different repo, and the source() method clones it. It just need to run a conan source . command to bring the code locally.

In theory the edit conanfile with the cmake_layout() could be enough to have the package in editable mode, I am not sure what is failing or what could be happening.

In order to further understand what could be failing, I think the best is to put the recipes together in a git repo that we can collaborate over, adding a small build.py script that automates the commands to reproduce (like conan editable add mypkg + conan install myconsumer). It would need to strip down some of the bits that are proprietary, like the git clone to the private repo, but probably it can be setup with simple hello-world projects like the one using the conan new cmake_lib -d name=pkg -d version=0.1 template. Do you think you could do that? Many thanks!

[memsharded]

Some extra hints regarding the used tools, just in case :)

• The from conans to import tools like Git and unzip is legacy. Please try to update to modern from conan.tools.xxx imports, they already exist in Conan 1.59
• The self.copy() is also legacy, it is recommended to move to the new from conan.tools.files import copy, also available in the Conan 1.59

[mangeMelon]

Quelques conseils supplémentaires concernant les outils utilisés, juste au cas où :)

• L' from conansimportation d'outils comme Gitet unzipest héritée. Veuillez essayer de mettre à jour les from conan.tools.xxximportations modernes, elles existent déjà dans Conan 1.59
• Il self.copy()s'agit également d'un héritage, il est recommandé de passer au nouveau from conan.tools.files import copy, également disponible dans Conan 1.59.

Thanks, I'm changing the tools right away. We built the conanfile from the examples and packages from the conancenter hence this mix.

Conanfile.py reference: complete and simple; with all the elements (but much simpler than the boost package). This reference conanfile will be appreciated by new people. In the documentation, definitions and examples, only the lines of code are visible, but its overall integration.

I wondered why conans.tools and sometimes conan.tools without really looking for the answer.

Otherwise I see cmake_layout() in two pages which is the correct one https://docs.conan.io/2/reference/tools/layout.html https://docs.conan.io/2/reference/tools/cmake/cmake_layout.html

[memsharded]

I wondered why conans.tools and sometimes conan.tools without really looking for the answer.

The migration guide in https://docs.conan.io/1/conan_v2.html might help with this. The idea is that recipes shouldn't use from conans at all, but move to the new from conan imports.

Otherwise I see cmake_layout() in two pages which is the correct one

Both pages are correct, the first one is just the introduction to the second one.

[mangeMelon]

So quick response.

The code of our modules is self-hosted.

One of the main ideas when using editables is that you do that because you have the conanfile.py together with the source code that you are packaging. In that sense is not usual to have a git.clone in the source() method of that recipe, because the recipe is already in the source repo.

What you say makes sense to me, but in https://docs.conan.io/1/reference/conanfile/methods.html#source shows a git clone in source() we just followed the example.

On the other hand, when creating our packages, I always found it frustrating to have to git clone to generate the package even though I have the sources on my machine. Not finding any other method other than download an archive.

I'm only trying to edit our packages and absolutely not the dependencies.

It is still possible to have an editable recipe for third-party code, where the source code is in a different repo, and the source() method clones it. It just need to run a conan source . command to bring the code locally.

In theory the edit conanfile with the cmake_layout() could be enough to have the package in editable mode, I am not sure what is failing or what could be happening.

I read and understand your sentences, but I do not interpret; can you detail?

In order to further understand what could be failing, I think the best is to put the recipes together in a git repo that we can collaborate over, adding a small build.py script that automates the commands to reproduce (like conan editable add mypkg + conan install myconsumer). It would need to strip down some of the bits that are proprietary, like the git clone to the private repo, but probably it can be setup with simple hello-world projects like the one using the conan new cmake_lib -d name=pkg -d version=0.1 template. Do you think you could do that? Many thanks!

I'm going to try to make a submission that covers our workflow, but I'm not sure I'll be able to communicate it to you before a week

[mangeMelon]

Hello @memsharded

thanks to another pair of hands, we managed to make our packages editable and consume them in another.

As I thought, our layout() method was incomplete. The file is attached.

Is it better to define the 'src_folder' as a parameter of 'cmake_layout(self)or through the variableself.folders.source`? Same for the 'build_folder'? in our case it depends on the 'build_type'

the folder generator depends in our case it depends on the OS and the 'build_type' (cross compiles Linux and windows), we have decided to have the OS / Build Type separation

and defining header files, library directory

Subsidiary question by following https://docs.conan.io/1/developing_packages/package_dev_flow.html#package-dev-flow I could test each part, but I encountered a problem when commanding conan package

cmd: $ conan package . --source-folder=tmp/source --build-folder=tmp/build --package-fold.er=tmp/package
Using lockfile: '/home/USER/git/moduleA/tmp/build/conan.lock'
Using cached profile from lockfile
ERROR: The usage of the 'conan package' local method is disabled when using layout(). Use 'export-pkg' to test if the recipe is packaging the files correctly or use the cpp.info.local object if you are going to use this package as editable package.

So I don't understand why, the package is not in editable mode, which restricts certain commands

---

I also wonder about the mechanics in the package() method; how cmake.install() successfully copied the files to the package directory. So I see that it works, but I haven't found a trace of explanation (Cimmerian's magic)

---

When searching for test on an option == None and why it doesn't work. I found this https://github.com/conan-io/conan/issues/9900 the user uses 'options_values = {"bar": None}this no longer exists and replace with 'default_options = {"bar": None} ?

Is self.options.get_safe("bar") supposed to return the value or None if not defined? In my case bar is set to None; so I test if self.options.get_safe("bar") == None: always false I suppose that 'get_safe' returns a character string "None" different from type(None)

I corrected it like this if (self.options.bar == "None"): conanfile_edit2.py.txt

[memsharded]

Happy that you fixed your layout() and made it work.

Is it better to define the 'src_folder' as a parameter of 'cmake_layout(self)or through the variableself.folders.source`? Same for the 'build_folder'? in our case it depends on the 'build_type'

The self.folders.source is not intended to be variable, as the sources for every recipe must be immutable. The relative location of the recipe with respect to the code should be always the same, and not an argument. The build folders are different, because they do depend on things like the build_type. But the source shouldn't The parametrization of the self.folders.build in layout() method should use the available parameters, like settings and options.

So I don't understand why, the package is not in editable mode, which restricts certain commands

The package command has been removed in Conan 2.0. layout() is simply not prepare to handle that command, because it is a "2.0-ready" feature. It is not really tied to being in "editable" mode, the layout() is also relevant for the local workflow, not only for editables.

Is self.options.get_safe("bar") supposed to return the value or None if not defined?

Yes, it will return None both if the values is not defined, but also if bar option doesn't exist.

In my case bar is set to None; so I test

You can't really set it to None, but only to the string literal "None" which is different behavior (more explicit in Conan 2.0). Asigning the string None anywhere like -o myoption=None is not recommended, the None values should be left for "not defined" use case, not for defined with the literal "None" string.