[question] Handling libc linker flags between applications and test packages

[kammce]

What is your question?

I'm working on https://github.com/conan-io/conan-center-index/pull/17528 and I'm not sure how to deal with test packages and applications that want to choose their libc specification.

With arm-gnu-toolchain for baremetal, you need an implementation for your libc. There are a couple of them out there but two of the provided compilers are the best for test packages because they get rid of the linker errors. The linker flags are --specs=nano.specs and --specs=nosys.specs. nano.specs defines implementations for memcpy, printf etc and nonsys.specs implements the lowest level C libraries for reading and writing to files and gathering memory for malloc. nosys.specs simply defines these as calls to abort/exit. It is meant to just fill the gaps and not to be used. This is great for test packages because it allows test packages to build without issue without each test package requiring special cross compiler flags in their cmake files. But my issue is figuring out how to disable this automatic injection for their applications. Here is what I'd like the user to be able to do:

conan build . -pr lpc4078 -pr arm-gcc-12.3 -s build_type=MinSizeRel

The lpc4078 profile is the target profile and contains information about the chip's architecture and os, etc. Here is what it looks like (the build type is a default for users if they want to override it):

[settings]
build_type=MinSizeRel
os=baremetal
arch=cortex-m4f

[options]
*:platform=lpc4078

[buildenv]
LIBHAL_PLATFORM=lpc4078
LIBHAL_PLATFORM_LIBRARY=lpc40

The build environment variables are for make, cmake and any other build system to be used to automatically link to the correct library. Note that cortex-m4f is not a arch that is currently available in the conan settings but is available in my user_settings. The platform option is something key to my ecosystem.

The arm-gcc-12.3 looks like this:

[settings]
compiler=gcc
compiler.cppstd=23
compiler.libcxx=libstdc++11
compiler.version=12.3

[tool_requires]
arm-gnu-toolchain/12.3

I distribute these both so users can select their target microcontroller and their target compiler. They could include() them both into a single profile if they wanted to for their project.

arm-gnu-toolchain will look at the arch flag and generate compiler & linker flags for the user based on their selection. The linker flags are the issue. The reason removing these is helpful is that nano.specs disables exception handling which I need for the software I'm building. I'd prefer to use picolibc or potentially another libc implementation. Here are the things I've tried:

Using Settings

compiler:
  gcc:
    libc: [null, "custom"]

This is what I tried. It made sense to me but the issue comes with how to affects packages. If I use this in my profile, none of my packages will have libc defined for them. I could recompile them for this, but the issue is that the libc type doesn't change how the binaries are generated. This setting only affects the final binary and how it is linked. So a setting for this doesn't really make sense, even though it would allow the compiler to choose correctly with a host profile.

Using Options

I tried options but they were never the correct choice for the tool package. Host profile options cannot find their way to the tool package and thus are not set correctly. So to make this work I'd have to add something like this:

conan build . -pr lpc4078 -pr arm-gcc-12.3 -pr:b custom-libc -s build_type=MinSizeRel

custom-libc looks like this:

include(default)

[options]
*:custom_libc=True

The default is included to get info like your native compiler, native arch, etc. That way I can just append my custom option to the profile. My only issue is that its really easy to forget that last flag. And it looks really strange. Forgetting to add this build profile is a silent failure which is a problem.

Using the CLI

This works, by adding -o:b custom_libc=True but adds even more text for the user and is error prone. It has the same issue as using the build options flag.

Using environment variables

I haven't tested this out yet but I believe it can work. I add the following to the lpc4078 profile:

[buildenv]
ARM_GNU_TOOLCHAIN_CUSTOM_LIBC="custom"

This way my compiler's toolchain.cmake file can do the following:

if(NOT $ENV{ARM_GNU_TOOLCHAIN_CUSTOM_LIBC} STREQUAL "custom")
  string(APPEND CMAKE_EXE_LINKER_FLAGS "--specs=nano.specs --specs=nosys.specs")
  message(WARNING "Appending nano and nosys libc specs to linker flags!")
endif()

This would work and alert the user at the very least, but still not my favorite option.

Which is the right path?

I'm not sure which is the right path so I'm asking the conan team for help here.

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[kammce]

I guess what this really comes down to, "can I inject flags for test packages and remove them for non-test package builds."

If my application, with a differing binary model to the applications already pre-built libraries, can I use -bmissing to build those dependencies and have their test packages build and properly link and also use the correct flags for my application.

new libc setting

My user_settings.yml should look like:

libc: [null, "custom"]

A suggested option was to use a new settings category such as "libc". Have my application opt into it by using settings = "...", "libc". This half way works. It works as I want with the profiles but it doesn't work for test packages. Unfortunately for test packages, the arm-gnu-toolchain can still see the global libc setting and applies it even though the test package does not opt into that setting, meaning the default is not set and the test package fails to link.

[kammce]

Here is what my version of the GNU ARM toolchain package_info method looks like:

    def final_libc(self):
        libc = self.settings_target.get_safe("libc")
        final_libc = None

        self.output.info(f"libc = {libc}")
        if libc:
            self.output.info("libc is alive! Getting newlib!")
            final_libc = self.settings_target.libc.get_safe("specs")

        self.output.info(f"final_libc = {final_libc}")

        return final_libc

    def package_info(self):
        self.cpp_info.includedirs = []

        bin_folder = os.path.join(self.package_folder, "bin/bin")
        self.cpp_info.bindirs = [bin_folder]
        self.buildenv_info.append_path("PATH", bin_folder)

        self.conf_info.define(
            "tools.cmake.cmaketoolchain:system_name", "Generic")
        self.conf_info.define(
            "tools.cmake.cmaketoolchain:system_processor", "ARM")

        self.conf_info.define("tools.build.cross_building:can_run", False)
        self.conf_info.define("tools.build:compiler_executables", {
            "c": "arm-none-eabi-gcc",
            "cpp": "arm-none-eabi-g++",
            "asm": "arm-none-eabi-as",
        })

        f = os.path.join(self.package_folder, "res/toolchain.cmake")
        self.conf_info.append("tools.cmake.cmaketoolchain:user_toolchain", f)

        if self._should_inject_compiler_flags:
            common_flags = self._c_and_cxx_compiler_flags
            self.conf_info.append("tools.build:cflags", common_flags)
            self.conf_info.append("tools.build:cxxflags", common_flags)
            self.conf_info.append("tools.build:exelinkflags", common_flags)
            self.output.info(f"C/C++ compiler & link flags: {common_flags}")

            libc = self.final_libc()
            if libc == "custom":
                self.output.info("Using custom libc implementation!")
            else:
                STUB_LIBC = ["--specs=nano.specs", "--specs=nosys.specs"]
                self.conf_info.append("tools.build:exelinkflags", STUB_LIBC)
                self.output.info(f"Using newlib libc: {STUB_LIBC}")

I also added some complexity to the libc for no other reason to but to see if it would do any difference. My current users settings look like:

libc:
  newlib:
    specs: [null, "custom"]

[kammce]

After speaking with the conan team on slack, I have come to the conclusion that this is not necessary. I was mistaken that test packages got built from dependencies if the prebuilt didn't exist. Since this is not the case, the problem does not exist.