[question] How to control which available compiler will be used when generating a profile by detection?

[Adnn]

We are using Conan to drive our CI processes.

The CI cloud provider offers different runners (available worker machines). On one of them, both gcc and clang are available. The runners are ephemeral (virtual machines starting from a clean default state on each run).

We run the build process via:

conan create conan-path/ ${package}/${version}@

This triggers the automatic detection and creation of the default profile. This automatically picks-up the gcc compiler.

• How could we detect another profile for clang?

(It would be impractical to create the clang profile manually, because we have no control over when the provider does compilers' version updates).

[memsharded]

Hi @Adnn

How could we detect another profile for clang?

In short you don't. You want to use your own profiles files, pre-created, and not rely on whatever Conan would be detecting on the system. Conan 2.0 Tribe already approved that the creation of default profile will not be automatic when such default profile is missing. Furthermore, the output of the Conan auto-detection will not be guaranteed to be stable either.

Are you using a free cloud provider that will upgrade your workers, including your compilers without you explicitly controlling it? That would be a nightmare, isn't it? A compiler upgrade is no joke, it can break easily break your builds, and things might take some time to be fixed, and you cannot have your production systems down because someone upgraded a compiler without your consent.

The strong recommendation is to use your own profiles, explicitly, and not to rely on Conan auto-detection, because that will always be fragile.

But if you have no other alternative, I would suggest:

• Defining your env-var CC=clang (or whatever), before running that, would probably select clang as the default compiler in the auto-detection.
• From Conan 1.38 you have profile templates, with .jinja extension, that you can put python Code there. You might use your custom logic, including reading env-vars to select the compiler you want (and hardcode the rest, to not rely on auto-detection): https://docs.conan.io/en/latest/reference/profiles.html#profile-templates

[Adnn]

Hi @memsharded, thank you for taking the time to answer this question.

Are you using a free cloud provider that will upgrade your workers, including your compilers without you explicitly controlling it? That would be a nightmare, isn't it? A compiler upgrade is no joke, it can break easily break your builds, and things might take some time to be fixed, and you cannot have your production systems down because someone upgraded a compiler without your consent.

Well, I agree with everything here. Sadly, our paid provider, Microsoft Azure Pipeline, did upgrade at least once the VS compiler version since we have been using them. Luckily nothing broke then, but we don't know if/when they'll do it again.

(As a side note, as soon as you rely on an external source, anything might break out of your control, and it is not like the average 10 developers company can do anything about it. For example, we have been using Conan Center provided recipes for some time, with a fixed Conan version ~1.2. One day, some recipes on Conan Center became incompatible with this version of Conan even though the recipe identifier was striclty identical. They just started raiseing exceptions because there was a new recipe revision, not retro-compatible, and all our automation broke. I think the bottom line is everyone is making non-consensual decisions that are impacting others).

Back on the topic, we are trying to already make sure our recipes will work with Conan 2, notably defining CONAN_V2_MODE (side note: even with this environment variable, the profile detection is automatic at the moment). So if it will disappear, we should probably not keep relying on it.

One drawback to disabling such automatic detection will be trying to get "unwilling" developers on board. This means now developpers have to take one more manual step of configuring a default profile, when it was automatic before. That "extra work that was not needed before we had Conan" will probably be something else we'll hear about during the regular "Complaining about Conan" sessions.

[memsharded]

For example, we have been using Conan Center provided recipes for some time, with a fixed Conan version ~1.2. One day, some recipes on Conan Center became incompatible with this version of Conan even though the recipe identifier was striclty identical. They just started raiseing exceptions because there was a new recipe revision, not retro-compatible, and all our automation broke. I think the bottom line is everyone is making non-consensual decisions that are impacting others).

Yes, this is documented in https://docs.conan.io/en/latest/introduction.html#stable, ConanCenter follows the policy of using latest version, it is the only possible way of moving things forward in this open community environment. Many fixes that many ConanCenter users need, for example, for cross-compiling, can only be possible with newer Conan versions. And it is not possible to change the identifiers, because you can't generate a boost2/version package for boost, just because you provided some cross-compiling capabilities to the recipe that were released in Conan 1.15. The consensus about the fact that ConanCenter should require the latest Conan version is quite large in the community of contributors and maintainers of ConanCenter.

In case you are interested, our recommendations to try to manage this are:

• Keep upgrading Conan, do not stay in a 1.2 which is years old now. We try hard to not break things, and when sometimes inevitably we do, we try to fix and release a patch as fast as possible.
• Use explicit pinned revisions of your dependencies, like zlib/1.2.11#revision, to avoid newer revisions to affect your code.
• Cache the dependencies you use from ConanCenter in your own ArtifactoryCE server, decoupling from ConanCenter after that (freeze them)
• Use a lockfile (with revisions enabled) to lock your dependencies

One drawback to disabling such automatic detection will be trying to get "unwilling" developers on board. This means now developpers have to take one more manual step of configuring a default profile, when it was automatic before. That "extra work that was not needed before we had Conan" will probably be something else we'll hear about during the regular "Complaining about Conan" sessions.

Yes, that is true. Until you release a product with a serious performance or security risk, because the automatic Conan profile detection detected and older compiler installed in that machine, or because the newest compiler upgrade was not detected correctly.

For production usage we have been recommending for a long time using profile files and the conan config install process to sync configuration, because it is not just about the profiles. Also the remotes that you want to use, or custom settings.yml, or custom hooks to validate something in the package binaries...

I have been there a few times, and this always remind me much of the SVN (or CVS) didn't xxxxxxx that now I need to do with Git, this is terrible/awful/much worse... while migrating to Git. I totally understand the pain. And we have thoroughly weighted pros and cons, and this is the reason we submitted the proposal to the tribe in https://github.com/conan-io/tribe/pull/24 (not a single downvote), after their initial request. We know it is not perfect, as always, there is no perfect solution, it is always a trade-off, and it will also slightly inconvenient some developers, but overall the benefits seems worth it.

[Adnn]

Thank you for the detailed answer @memsharded

Just a precision:

Yes, that is true. Until you release a product with a serious performance or security risk, because the automatic Conan profile detection detected and older compiler installed in that machine, or because the newest compiler upgrade was not detected correctly.

In the context of this issue, we are using CI to produce all releases (there are indeed many potential risks if anything built directly on a developer machine can leave the "safe circles" of the organization ; ), so the specific remark regarding dev environments was about making developpers life one command shorter with automatic detection.

This is a strong argument though for not having automatic detection in the CI environments, thanks!