[feature] Add shutil to conan profile render context

[rols121]

What is your suggestion?

My usecase is to overcome the naming differences for cross compilers, particularily deviations from standard compiler name triplet (machine-vendor-operatingsystem). The aarch64 compiler downloaded from the ARM website downloads has executable names that following the aarch64-none-linux-gnu- host name, but those installed via apt come with a aarch64-linux-gnu- host name, not to mention that if you want a non-default version from apt, then it appends the version number to the end of the executable name, e.g. aarch64-linux-gnu-gcc-12. I realise this can be solved with symlinks etc. but it would be great if this can be hidden from the final user by adding logic to the conan profiles that allows for detecting the existance of a compiler from a list of possible names, e.g.

[settings]
os=Linux
arch=armv8
compiler=gcc
compiler.version=12
compiler.libcxx=libstdc++11

[conf]
tools.cmake.cmaketoolchain:generator="Ninja"
tools.cmake.cmaketoolchain:system_processor=arm64
tools.cmake.cmaketoolchain:extra_variables={"CMAKE_TRY_COMPILE_TARGET_TYPE": "STATIC_LIBRARY"}

[buildenv]
{% set cross_compiler_options = ["aarch64-linux-gnu-gcc", "aarch64-linux-gnu-gcc-12", "aarch64-none-linux-gnu-gcc", "aarch64-none-linux-gnu-gcc-12"] %}
{% set CC_ENV = os.getenv("CC") %}
{% if CC_ENV %}
CC={{ CC_ENV }}
CXX={{ CC_ENV | replace("gcc", "g++") }}
{% else %}
    {% for cross_compiler_option in cross_compiler_options if shutil.which(cross_compiler_option) %}
CC={{ cross_compiler_option }}
CXX={{ cross_compiler_option | replace("gcc", "g++") }}
    {% endfor %}
{% endif %}

This can be achieved by adding shutil to the profile render context (diff below), or allowing the context to be modified from the conanfile.py itself (probably better).

diff --git a/conan/internal/api/profile/profile_loader.py b/conan/internal/api/profile/profile_loader.py
index 8b36ef61f..f9706c1c4 100644
--- a/conan/internal/api/profile/profile_loader.py
+++ b/conan/internal/api/profile/profile_loader.py
@@ -1,4 +1,5 @@
 import os
+import shutil
 import platform
 from collections import OrderedDict, defaultdict

@@ -154,6 +155,7 @@ class ProfileLoader:
         file_path = os.path.basename(profile_path)
         context = {"platform": platform,
                    "os": os,
+                   "shutil": shutil,
                    "profile_dir": base_path,
                    "profile_name": file_path,
                    "conan_version": conan_version,

Or this maybe available somehow already and my documentation reading skills are bad... The documentation for conan is great by the way, so thank you for that.

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[czoido]

Hi @rols121,

Thanks a lot for your suggestion and your kind words. I'll mark this as look-into so our team can consider it. In the meantime, perhaps you could use os.system(f"command -v {cross_compiler_option}") == 0 instead of shutil?

[rols121]

Thanks @czoido, yes, I have settled on the following for now, which works, but is a little less succinct:

{% set CC_ENV = os.getenv("CC") %}
{% set CXX_ENV = os.getenv("CXX") %}
{% if CC_ENV and CXX_ENV %}
CC={{ CC_ENV }}
CXX={{ CXX_ENV }}
{% else %}
    {% set cmd = 'where %s 2> nul' if (platform.system() == 'Windows') else 'which %s 2> /dev/null' %}
    {% for cross_compiler_option in cross_compiler_options if (os.popen(cmd | format(cross_compiler_option[0])).read() and os.popen(cmd | format(cross_compiler_option[1])).read()) %}
CC={{ cross_compiler_option[0] }}
CXX={{ cross_compiler_option[1] }}
    {% endfor %}
{% endif %}

[memsharded]

Hi @rols121

There is the detect_api.detect_gcc_compiler(compiler_exe="gcc") available that you might use in your for loop, and you won't need neither shutil nor os.popen. Could you please give that a try? I think it would be a more built-in approach.

I'd also like to warn that it is possible that those different compiler names, from different machines/systems/platforms are actually not the exact same compiler. They might be aarch64, gcc 12 based compilers, but they might have been built differently, with different settings, different defaults, etc. Maybe it is not likely that they produce binary incompatibilities, but it is not impossible either. @jcar87 has been bitten by this before, very similar scenario, and it might be that modeling those as different actual compilers might make sense (extra custom settings, or something like that)