[question] Conan cross compilation - my experience

[derived-coder]

Hi Barbarians,

I want to give you some feedback about cross compilation with conan.

1) In my opinion, the CLI interface for cross compilation is too complicated.

conan create . 1.0.0@ -pr:h gcc-9.0-linux-armv8-release -pr:b gcc-9.0-linux-x86_64-release

Why not using one profile?

conan create . 1.0.0@ -pr gcc-9.0-linux-x86_64_to_armv8-release

I do not know what the reason was to use two profiles, but I find it more complicated to use.

Possible solution: Use one profile, but under the hood, because of profile composition, you still use two profiles. E.g:

# gcc-9.0-linux-x86_64_to_armv8-release profile
[host]
include(gcc-9.0-linux-armv8-release)
[build]
include(gcc-9.0-linux-x86_64-release)

2) The other thing is the naming: host and build is used by conan. I find it not intuitive what is meant with it. Is build now the target architecture or host? Not sure.... I also see it from time to time that people asking this question in slack. But the naming should be so good, that you don't have to look in the documentation and know immediately what it means. I prefer the yocto naming: host and target

3) Writing recipes which handle proper cross compilation are quite hard, IMHO. You have these: settings_target, settings_host, setting_build which are implicit available, through a logic which is not easy understandable. Then there is also the --build-require which seems to play also a role during cross compilation...It's too much. Maybe the docs are not helping good enough, or maybe I am too dumb.

4) There should be cross compilation enabled in CCI by default, at least for linux x86_64 to ARM. The whole automotive industry is moving to embedded devices, where 1st class cross compilation support would be very handy. My feeling is that this section is not getting the attention which it should get from conan. There is a lot of potential for conan in this sector with proper cross compilation support.

[memsharded]

Hi @derived-coder

I think you are missing the history here:

1. The 2 profiles are needed to be defined as an opt-in, because it is the only way to be able to apply the new behavior without breaking the old behavior. Remember that by our 1.X commitment, we cannot break. The legacy behavior is using os_build and arch_build, that happens when you specify 1 profile. But we learned that model is quite bad, but even if it is bad, we cannot break it. Applying the 2 profiles allows not breaking and introducing the new feature.

Conan 2.0 already implemented using always 2 profiles. Even if you don't specify it, they will be both defined and using the "default" one

2. We didnt invent the naming. We chose, recommended by users, the standard GNU naming, please check it. Even if I like the yocto one, the GNU cross-compile naming (that is, the flags passed to gcc compiler follow this rule) is older and more spread than yocto.

3. Yes, cross compilation is difficult, and complex. Those settings are needed for some recipes and toolchains, and the definitions follow the naming clarified in point 2). There is little that Conan can do there, besides defining them. You need to understand the GNU contexts first, then now that Conan will provide the settings for each one.

Regarding the --build-require when you are creating a package that should be more understandable when you understand the build and host context. If you are creating a tool package, which is going to be a toolchain package in the "build" context, applied as a --build-require to other Conan packages, Conan needs to be aware of that. If you don't pass the --build-require argument, Conan will treat it as a "host" package, like a regular library, and pass it the "host" settings instead of the build ones. Conan needs to be aware somehow that it is a toolchain/build-require package, it is impossible to do the right thing otherwise.

4. Yes, we will eventually add some support in ConanCenter CI for cross-build (actually, it is already cross building from Mac x86_64 to ARM (M1), and it is working good). Other users are contributing fixes to recipes for cross-building.

Conan cross-building and the 2 profiles/contexts model has been getting attention and is already being succesfully used by users. It requires knowing the 2 profiles model, and Conan 2.0 will improve the UX with better defaults and better build-system integrations support, but the core principles and concepts will be the same.

[derived-coder]

@memsharded

1. I understand that conan already uses under the hood two profiles and thats fine. But what do you think about my compromise for a new simplified CLI for cross compiling?

2. Okay understand.

3. I still don't get it 😇 Maybe it would be helpful to show/create a template recipe for a toolchain (once and for all) and how it should be created. I asked for it here : https://github.com/conan-io/conan/issues/10947 Remark: the standard use case for a toolchain is, to just download and extract and archive and setting up env variables, nothing will be compiled.

[lzptr]

@memsharded

First of, I am learning to use conan now, and I really enjoy it. Thank you very much for it!

I am trying to understand cross compiling with conan and I am too confused about the host and build profile naming. And I have to argue, that the naming is entirely spot on. GCC uses a triple notation for the cross compilation:

"There are three system names that the build knows about: the machine you are building on (build), the machine that you are building for (host), and the machine that GCC will produce code for (target). When you configure GCC, you specify these with --build=, --host=, and --target="

...

• If build, host, and target are all the same, this is called a native.

• If build and host are the same but target is different, this is called a cross.

• If build, host, and target are all different this is called a canadian (for obscure reasons dealing with Canada’s political party and the background of the person working on the build at that time).

• If host and target are the same, but build is different, you are using a cross-compiler to build a native for a different system. Some people call this a host-x-host, crossed native, or cross-built native.

• If build and target are the same, but host is different, you are using a cross compiler to build a cross compiler that produces code for the machine you’re building on. This is rare, so there is no common way of describing it. There is a proposal to call this a crossback.

Source: https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html

As mentioned in this blog post here: https://www.linuxquestions.org/questions/programming-9/gcc-clarity-on-difference-between-build-host-and-target-849125/, the host often refers to the case, were you need to cross compile gcc for another (proxy) host that compiles the final application that should run on the target.

An example would be:

• Build: linux_x86 -> has native gcc, can cross compile gcc
• Host: linux_raspberry_arm -> has no cross compiler for risc-v but needs to cross compile
• Target: baremetal_risc_v -> runs the final application

As I understand it it would be:

• build = where am I compiling the compiler
• host = where the compiler will run
• target = what code will the compiler produce.

Now I know that the naming conventions for these are not used everywhere consistently, but as far as microcontroller systems go, I never heard anyone refer to it as the host. Target is definitely more often used when cross compiling.

I hope this helps, and it would be pretty nice if the name of the host profile file could be changed to target profile :)

[derived-coder]

You have my vote, it is counter intuitive to use host for the actually target system.

[derived-coder]

@jcar87 Are you working on my 1) suggestion?

[memsharded]

Conan 2.0 made the build profile not necessary in the command line and defaulted to -pr:b=default (same default as host profile).

The naming of the build-host-target is also well stablished, keep mostly aligned to GNU, even if not the most intuitive for all users coming from different backgrounds, but it is being maintained, this naming is not going to change. Thanks for the feedback!