[question] Confused on interaction between layout() and source() with Git

[JackRenner]

What is your question?

I am running into an interesting problem with the interaction between layout() and source() with Git. When using the cmake_layout + root/subproject syntax outlined here, I am seeing different behavior when using local install/build commands vs create. The key issue I believe is that layout() is setting the self.source_folder to be the subproject folder, which locally works, but occurs BEFORE source() is run during create, which changes the relative paths (the repo is cloned into a fake/empty subproject folder in the cache, and ends up telling CMake to use the wrong CMakeLists.txt during build(). (NOTE: these CMakeLists.txt are disconnected and not within the same tree)

What I am looking for (I think) is a way to tell conan that the source dir is the project root, but CMake that its source is "unique-subproject". After a few hours of hunting through the documentation and trying various iterations of custom layouts/cmake flags, I am hoping that someone else has had this problem before. If so, any guidance would be much appreciated!

My functions are as follows:

generators = "CMakeToolchain", "CMakeDeps"

def export(self):
  git = Git(self, self.recipe_folder)
  scm_url = git.get_remote_url()
  scm_commit = git.get_commit()
  update_conandata(self, {"sources": {"commit": scm_commit, "url": scm_url}})

def source(self):
  git = Git(self)
  sources = self.conan_data["sources]
  git.clone(url=sources["url"], target=".", args=["-c", "http.sslVerify=false"]
  git.checkout(commit=sources["commit"])

def layout(self):
  self.folders.root = ".."
  self.folders.subproject = "unique-subproject"
  cmake_layout(self)

The repo is structured like this (simplified):

repository/ ├── conanfile.py ├── CMakeLists.txt ├── src/ ├── unique-subproject/ │ ├── CMakeLists.txt │ ├── conanfile.py │ ├── src/

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @JackRenner

Thanks for your question.

Quick question: I understand that the conanfile.py you provide is the one inside unique-subproject isn't it?

I think this is the same use case that we have in this test: https://github.com/conan-io/conan/blob/d144ef2e27dcd65917898ad034c4b416c5713eab/test/functional/tools/scm/test_git.py#L844

What you might be missing is an extra move of the source after the checkout:

    from conan.tools.files import load, copy, save, update_conandata, move_folder_contents

                def source(self):
                    git = Git(self)
                    sources = self.conan_data["sources"]
                    git.clone(url=sources["url"], target=".")
                    git.checkout(commit=sources["commit"])
                    # Layout is pkg/pkg/<files> and pkg/common/<files>
                    # Final we want is pkg/<files> and common/<files>
                    # NOTE: This abs_path is IMPORTANT to avoid the trailing "."
                    src_folder = os.path.abspath(self.source_folder)
                    move_folder_contents(self, src_folder, os.path.dirname(src_folder))

Please try that and let us know.

[JackRenner]

Yes, the conanfile.py is from unique-subproject

It looks like move_folder_contents cannot account for the fact that the self.source_folder already exists since it is created before source() ever runs.

I did figure out another way that seems consistent. I am curious what your thoughts are if this should be the advertised way of handling git clones in source() when using a nested conanfile.py ?

def source(self):
  src_folder = os.path.abspath(self.source_folder)
  os.chdir(os.path.join(src_folder, "..") ## adjust the path for cloning without modifying self.source_folder
  shutil.rmtree("unique-subproject") ## remove the conan-generated self.source_folder to allow git clone to work
  git = Git(self)
  sources = self.conan_data["sources]
  git.clone(url=sources["url"], target=".", args=["-c", "http.sslVerify=false"]
  git.checkout(commit=sources["commit"])