[question] How to declutter the test package's source repository after building it?

[PauloCarvalhoRJ]

Greetings,

After building and running a test package, I noticed the source repository is filled with build artifacts (CMake cache, Visual Studio intermediary files, etc.). By reading the following in https://docs.conan.io/2/tutorial/creating_packages/test_conan_packages.html, I understand that the build artifacts are, by default, generated in the test package's source directory. I don't think this is desirable, since a good practice is to leave the source repository (e.g. Git) untouched. If unavoidable, is there a way to cleanup the test package's source repository?

much obliged,

PC

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[czoido]

Hi @PauloCarvalhoRJ,

Thank you very much for your question. Typically, the build files for the test_package are generated in a separate build folder inside the test_package directory. To manage these build artifacts, you have a couple of options:

• Use the tools.cmake.cmake_layout:test_folder configuration to put the files out of the test_package folder, If that value is $TMP, Conan will create and use a temporal folder.

This can be defined in your profile for example like this:

include(default)
[conf]
tools.cmake.cmake_layout:test_folder=$TMP

• Add the build folder to the .gitignore file: This will ensure that the build artifacts are not tracked by Git, keeping your source repository clean.

• Remove the build folder in the conanfile.py: You can modify the test() method in your conanfile.py to delete the build folder right after testing the package. This will clean up the build artifacts automatically.

Hope this helps

[PauloCarvalhoRJ]

Hello, @czoido ,

I tried that and Conan still pollutes my source repository with build artifacts. Here's my default profile file:

[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=14
compiler.runtime=dynamic
compiler.version=194
os=Windows

[conf]
tools.cmake.cmake_layout:test_folder=build_files

What am I doing wrong?

best,

PC

[czoido]

Hi @PauloCarvalhoRJ,

If you want to put the build artifacts out of the source repository, specify an absolute path that it's outside it instead of a relative path as you are doing:

[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=14
compiler.runtime=dynamic
compiler.version=194
os=Windows

[conf]
tools.cmake.cmake_layout:test_folder=/absolute/path/to/build_files

Also, if you set the value to $TMP Conan will take care of creating a temporary folder and using that for the build.

...
[conf]
tools.cmake.cmake_layout:test_folder=$TMP

[PauloCarvalhoRJ]

Hello, @czoido ,

It still doesn't work. I changed it to C:/conan_tmp, C:\\conan_tmp and C:\conan_tmp (this directory exists). I got the very same results: source repository cluttered with build artifacts. The $TMP value is similarly innefective. To rule out the possibility of Conan be using another profile file, I entered gibberish into it and Conan issued an error, so the profile file I'm editing is the same that Conan is using.

thank you,

PC

[czoido]

Hi @PauloCarvalhoRJ,

For me it works fine. We also have this covered in the test suite for Win, Mac and Linux

Which Conan version are you using? This conf was introduced in the 2.2.0 version. If you are using >=2.2.0 can you do something like this?

create the profile myprofile with this content:

include(default)
[conf]
tools.cmake.cmake_layout:test_folder=C:\conan_tmp

then:

conan new cmake_lib -d name=mypkg -d version=1.0 --force
conan create . -pr=myprofile

Check that it was built in C:\conan_tmp

[PauloCarvalhoRJ]

Hello, @czoido ,

Sorry, Conan here is 2.3.2. I edited a new profile file like you suggested and used it in a conan create . command still for no avail. Perhaps there's something wrong in the recipes. Please, find attached the source of zlib I'm building, including its test package.

cheers,

PC zlib_package.zip

[czoido]

Hi @PauloCarvalhoRJ,

Thanks for sharing, I can see the problem there. In order for the conf to take effect you have to use the cmake_layout, otherwise it will continue to pollute your test folder. I would suggest to update your recipe as I can see several things that can be improved, you can check the Conan Center Index recipe for zlib for reference: https://github.com/conan-io/conan-center-index/blob/master/recipes/zlib/all/conanfile.py maybe you could align a bit more with that to simplify yours.

Hope this helps.

[PauloCarvalhoRJ]

In order for the conf to take effect you have to use the cmake_layout.

So, I believe the docs are due an update with that info, then. Either this or a Conan warning in CLI like "[conf] is defined in profile, but the recipe does not have a cmake_layout() call." Whithout that, I was left scratching my head over what I was doing wrong. Just a thought.

[PauloCarvalhoRJ]

update your recipe as I can see several things that can be improved

I agree, However, I wish I could afford the time to rewrite all the recipes I have in hands (it's a large project). The most I can do right now is to migrate existing code so the recipes work in Conan 2.

[czoido]

In order for the conf to take effect you have to use the cmake_layout.

So, I believe the docs are due an update with that info, then. Either this or a Conan warning in CLI like "[conf] is defined in profile, but the recipe does not have a cmake_layout() call." Whithout that, I was left scratching my head over what I was doing wrong. Just a thought.

Sorry for that, we will try to clarify in the docs as I don't think it's possible to raise a warning when using the conf.

I would suggest you to try the other alternatives I mentioned above.

[PauloCarvalhoRJ]

So, I added cmake_layout(self, src_folder=".") to the layout() callback of the recipe (test package's). When doing a conan create . -pr=profile_uncluttered_tests on the package under test (zlib), I got this:

======== Testing the package: Building ========
zlib/1.2.11 (test package): Calling build()
zlib/1.2.11 (test package): Running CMake.configure()
zlib/1.2.11 (test package): RUN: cmake -G "Visual Studio 17 2022" -DCMAKE_TOOLCHAIN_FILE="generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="C:/GR_DEV/GammaRay/conan_recipes/zlib/1.2.11/test_package" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" "C:/GR_DEV/GammaRay/conan_recipes/zlib/1.2.11/test_package"
-- Using Conan toolchain: C:/GR_DEV/GammaRay/conan_recipes/zlib/1.2.11/test_package/conan_toolchain.cmake
-- Conan toolchain: CMAKE_GENERATOR_TOOLSET=v143
-- Conan toolchain: C++ Standard 14 with extensions OFF
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22631.
-- Conan: Target declared 'zlib::zlib'
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: C:/GR_DEV/GammaRay/conan_recipes/zlib/1.2.11/test_package

zlib/1.2.11 (test package): Running CMake.build()
zlib/1.2.11 (test package): RUN: cmake --build "C:\conan_tmp\msvc-194-x86_64-14-release" --config Release
Error: could not load cache

I find it odd that CMake.build() tries to run the build in the build directory specified in the profile file while CMake.configure() just above the former in the build() callback, outputs the build files to the source repository, hence the error. BTW, here's the build() callback of the test package's recipe:

    def build(self):
        cmake = CMake(self)
        cmake.configure()   # writes build files to the source repository (wrong)
        cmake.build()       # calls cmake --build ... on the correct directory --> ERROR!

best, PC

[memsharded]

I am not sure what could be failing, I am trying to reproduce without success:

conan new cmake_lib -d name=mypkg -d version=0.1
conan create . -c tools.cmake.cmake_layout:test_folder=C:/conan_tmp

it works fine, and it only adds the CMakeUserPresets.json file in the test_package folder, but not all the other files

[PauloCarvalhoRJ]

Hello, @memsharded ,

Please, try the source I posted further above to reproduce the issue.

thanks,

PC

[PauloCarvalhoRJ]

Hi, @czoido and @memsharded ,

I cleaned everything up and rebuilt evertything from scratch. The source repository is no longer being pulluted (except for the CMakeUserPresets.json which is a minor nuisance that I can add to the .gitignore file). I believe there was something corrupted in the Conan cache that was causing the incorrect behavior after I applied @czoido's fix. Sorry about that.

thanks,

PC