[feature] Allow specifying the linker in tools.build:compiler_executables

[jwillikers]

What is your suggestion?

There are use cases for using a linker other than the system default, such as cross-compilation and link-time optimization. Currently, users must specify the linker through build-system dependent environment variables using the [buildenv] section.

Build systems do not uniformly make use of the LD environment variable to define the linker, which leads to confusion. Meson, for instance, uses different environment variables than the common LD environment variable, which has lead to the issues #13824 and #14028. Furthermore, build systems may provide more robust methods of specifying the linker than using environment variables. Meson, for instance, allows defining the linker in cross and native files, which are already generated by Conan. This is described here. Allowing a linker to be specified via tools.build:compiler_executables would unify specification of the linker and reduce confusion and duplication.

Meson allows defining a linker for both the C and C++ languages, so I would propose that we do the same in tools.build:compiler_executables, i.e. c_ld and cpp_ld. CMake and Autotools toolchains should be able to make use of these values as well.

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @jwillikers

Thanks for your suggestion. I think this proposal makes sense, and the possibility of defining the linker for C and C++ as suggested and aligned with the compilers, also makes sense.

I'd like to see how the info is passed to the different build systems. If you are willing to give this feature a try, please let us know, otherwise it might need to wait a bit, as there are already too many higher priorities ongoing. Thanks!

[jwillikers]

I might find time before the team gets around to it. I'll update if / when I start work on it.

[jwillikers]

I think I have a better understanding of how to best solve this now. I think that instead of using tools.build:compiler_executables, the linker should instead be specified with a dedicated tools.build:linker conf option. Meson and CMake require the linker to be specified in a specific way which does not include the actual path to the linker in many cases.

CMake 3.29 introduced the CMAKE_LINKER_TYPE variable which can be used to configure the linker. This can be set to values such as BFD, GOLD, LLD, and MOLD. Meson has functionality to set the linker as described here, which specifies the linker in cross or native files via variables like c_ld and cpp_ld. Meson also takes values like bfd, gold, lld, and mold. Notably, Meson requires that an executable be specified when using the msvc and clang-cl compilers.

Given the CMake and Meson ways of setting the linker, compiler_executables doesn't seem like the best fit, which is why I'm proposing tools.build:linker instead. Implementing support for bfd, gold, lld, and mold on non-windows platforms should be pretty straightforward for CMake and Meson as these same values can be easily be inserted using their respective toolchains. sold, i.e. mold for macOS, may only be directly supported by CMake as I don't see it listed in the Meson's linker ids table. For CMake versions prior to 3.29, it's still possible to trivially support these linkers by passing the linker flags directly like as follows.

if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.29.0")
  set(CMAKE_LINKER_TYPE MOLD)
else()
  list(APPEND CMAKE_EXE_LINKER_FLAGS_INIT -fuse-ld=mold)
  list(APPEND CMAKE_MODULE_LINKER_FLAGS_INIT -fuse-ld=mold)
  list(APPEND CMAKE_SHARED_LINKER_FLAGS_INIT -fuse-ld=mold)
endif()

Conan will probably also want to take into account compiler versions and linker support as is done by CMake and Meson. I ran into this with the mold linker and GCC 11 on Ubuntu 22.04. While GCC 11 on Ubuntu 22.04 is patched to support mold as the linker, it wasn't officially supported by upstream until GCC 12. Thus, both Meson and CMake only allow using mold as the linker when GCC is version 12 or newer. Refer to the discussion in CMake issue 25748 for further details. Anyways, Conan should probably implement a similar check of its own for this situation.

Limitations of the current solution I've proposed here include the inability to override the actual linker executable, which I think some consumers may still wish to do and the fact that this only addresses CMake and Meson, but not Autotools or msbuild.

[jcar87]

From my comment in #:

Until the most recently released CMake version 3.29, there is no way to tell CMake which linker to use - bear in mind that CMake uses the compiler to perform the link step, rather than invoking the linker directly.

We are considering how to best integrate the new CMAKE_LINKER_TYPE

In the meantime, I suspect the easiest way to achieve a different linker globally across all generators, is to use the tools.build:exelinkflags and tools.build:sharedlinkflags to pass -fuse-ld=lld as flags - and that should do the trick, and this should result in those variables you mention above @jwillikers , CMAKE_EXE_LINKER_FLAGS_INIT and the likes.

Alternatively, if you're only targetting CMake, you can use a custom toolchain file that contains the line add_link_options(-fuse-ld=lld) and specify the path to this toolchain file in the tools.cmake.cmaketoolchain:user_toolchain conf.

There are use cases for using a linker other than the system default, such as cross-compilation and link-time optimization.

The compiler is "hardwired" to find the linker in some specified way. When cross-compiling, this will try and find the linker that is part of the same toolchain, and should not use the system default at all.

Limitations of the current solution I've proposed here include the inability to override the actual linker executable,

At least for gcc, -fuse-ld=abc will look for the abc executable in the PATH. So it is possible to override the path to the linker, by adding -fuse-lld=lld to tools.buiild:exelinkflags and tools.build:sharedlinkflags, as well as prepending to the PATH environment variable in buildenv in the profile.

[hoyhoy]

What works now is compiling llvm with -DCLANG_DEFAULT_LINKER:STRING=lld. I also set CLANG_CONFIG_FILE_SYSTEM_DIR so I can override any clang setting including the default linker with a cfg file.

I override LD and LINKER to lld in [buildenv] because meson and autotools don't use the clang the clang binaries directly to link. I also ended up hand-editing the libtool script for MacOS.

{% set profile = {
    "build_type": platform.os.getenv("CONAN_BUILD_TYPE", default="Debug"),
    "settings": { "os": current_os, "arch": platform.machine() },
    "options": {},
    "configuration": {},
    "replace_requires": [],
    "replace_tool_requires": [],
    "build_configuration": {},
    "environment": {}
  }
%}

{% set llvm_root = platform.os.getenv("LLVM_ROOT", default="/opt/llvm" ~ compiler.version) %}
{% set llvm_bin = llvm_root ~ "/bin" %}
{% set llvm_lib = llvm_root ~ "/lib" %}
{% set llvm_system = platform.system().lower() %}
{% set compiler.system = llvm_system %}
{% set compiler.llvm_root = llvm_root %}
{% set compiler.llvm_bin = llvm_bin %}
{% set compiler.lib_dir = llvm_lib %}
{% set compiler.include_dir = llvm_root ~ "/include" %}
{% set compiler.cc = llvm_bin ~ "/clang" %}
{% set compiler.cxx = llvm_bin ~ "/clang++" %}
{% set compiler.ld = llvm_bin ~ "/" ~ llvm_linker %}
{% set compiler.ar = llvm_bin ~ "/ar" %}
{% set compiler.as = llvm_bin ~ "/as" %}
{% set compiler.nm = llvm_bin ~ "/nm" %}
{% set compiler.ranlib = llvm_bin ~ "/ranlib" %}

{# these should all be set automatically via conan tools.* settings... #}
{# but, only the compiler settings work #}
{# there are separate CMake variables for these as well.. #}
{# and separate b2 settings, none of which are set via conan #}
{# which require custom profile hacks like this that shouldn't be necessary #}

{% set _ = profile.environment.update({
  "CPATH": compiler.include_dir,
  "CC": compiler.cc,
  "CXX": compiler.cxx,
  "LD": compiler.ld,
  "AS": compiler.as,
  "AR": compiler.ar,
  "NM": compiler.nm,
  "LLVM_ROOT": compiler.llvm_root,
  "RANLIB": compiler.ranlib,
  "LINKER": compiler.ld,
  "LD_LIBRARY_PATH": compiler.libpath,
  "LIBPATH": compiler.libpath,
  "VERBOSE": "1"
  })
%}

[buildenv]
{% for environment_variable,value in profile.environment.items() -%}
{{environment_variable}}={{value}}
{% endfor %}

It'd be great if the conan tools settings just worked across all build toolchains (including exelinkflags).

[rdong8]

This would be really great. I've noticed that setting CMAKE_LINKER_TYPE doesn't do anything when using Conan, I guess the linker is being set somewhere else in the toolchain file.

[jcar87]

This would be really great. I've noticed that setting CMAKE_LINKER_TYPE doesn't do anything when using Conan, I guess the linker is being set somewhere else in the toolchain file.

To use mold as the default linker, you can consider: https://github.com/conan-io/conan-center-index/tree/master/recipes/mold

[rdong8]

I'm not sure if I'm doing it right, but I seem to be getting an error with this:

BUILD_DIR=build \
        conan \
                install . \
                -b missing \
                -s build_type=Release \
                -s "&:build_type=Debug"

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=23
compiler.libcxx=libc++
compiler.version=18
os=Linux
&:build_type=Debug
[tool_requires]
*: mold/[*]
[conf]
tools.build:exelinkflags=['-fuse-ld=mold']
tools.build:sharedlinkflags=['-fuse-ld=mold']
[buildenv]
CC=clang
CXX=clang++

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=23
compiler.libcxx=libc++
compiler.version=18
os=Linux
[tool_requires]
*: mold/[*]
[conf]
tools.build:exelinkflags=['-fuse-ld=mold']
tools.build:sharedlinkflags=['-fuse-ld=mold']
[buildenv]
CC=clang
CXX=clang++

======== Computing dependency graph ========
Graph root
    conanfile.py: /home/rd8/Projects/cpp_project/conanfile.py
Requirements
    boost/1.85.0#0146e46a0e32fbb44e7189f25c193dae - Cache
    zlib/1.3#06023034579559bb64357db3a53f88a4 - Cache
Build requirements
    mold/2.33.0#8a8d5804d73c875342480c9636a0c83c - Cache
    zlib/1.3#06023034579559bb64357db3a53f88a4 - Cache
Resolved version ranges
    mold/[*]: mold/2.33.0
    zlib/[>=1.2.11 <2]: zlib/1.3
ERROR: There is a cycle/loop in the graph:
    Initial ancestor: mold/2.33.0
    Require: mold/2.33.0
    Dependency: mold/2.33.0
make: *** [Makefile:47: conan-deps] Error 1