[feature] Feature flags

[IsaacDynamo]

What is your suggestion?

Hi,

Other package managers such as Cargo and vcpkg have the concept of feature flags. Feature flags can be used to enable optional functionality and dependencies, and feature flag sets from all dependents are resolved with set union semantics.

I looked through the documentation but couldn't find this functionality.

I tried to model feature flags with options and with a custom cpp_info.set_property() + generate() but both didn't workout. The options attempt was unsuccessful because features flags don't unify and snap to the features set of the first dependent, as explained in the FAQ With generate() it was possible to scan through all dependencies and unify there feature flags, that where set with a custom cpp_info.set_property(), but with this attempt wasn't possible to encode optional dependencies because the flags of the dependents are not yet known in the requirements() call.

I think feature flags could be a nice addition to Conan as a more restricted alternative to options, because flags can only be additive. But this restrictions allows to give better guarantees about composability, because the flags can be unified across the whole dependency tree.

In the mean time I would love to hear suggestions how to get similar behavior with the current version of Conan.

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @IsaacDynamo

Thanks for your suggestion.

I totally agree that set_property() is not a viable mechanism for this.

But I think that options are, let me explain the rationale and where we come from:

attempt was unsuccessful because features flags don't unify and snap to the features set of the first dependent

You cite this as an inconvenience to use options as feature flags. But this is the way that options were managed in Conan 1, and it was removed in Conan 2 because it had many problems, it really didn't work either.

One of the reasons it didn't work is because Conan users do not want "additive-only" feature flags. Many of them really request the possibility of removing or deactivating feature flags, and being able to force that, for many reasons from security, to resources, etc. So an "addition-only" mechanism doesn't seem what the user wants. We have lots of experience in ConanCenter, with tons of packages using with_xxx and without_yyy options as feature flags.

The moment that feature flags need to be both enabled and disabled, then it happens that there can be conflicts, and there will be different packages in different parts of the dependency graph setting contradictory option values, and those conflicts need to be resolved.

And finally, there is a very important reason: the package managers you cite above are mostly build-from-source package managers, that always build the full dependency graph from source. But Conan is also focused on managing binaries, and a very frequent use case is that packages are built in a decentralized (and parallel) way when possible. Let me put a very simple example: You have app -> libb -> liba. Now assume that liba has some feature flags, that by default they are disabled, and libb enables them in its recipe. Conan users want to be able to build liba binaries first, in an independent pipeline, lets say for the different platforms they support (Windows, Linux, etc), and upload the binaries to their server repo. If they do that, they will build the binaries with no features. Then the pipeline for libb is fired and it will fail, because it needs those features of liba enabled. In no way liba has to be built while building libb, even if Conan allows it with --build=missing, this is not good practice for CI. Because liba can be Qt that takes a very long time to build. So Qt is not going to be built again from source when some consumer requires it with some different feature flags.

So the approach to this is very simple: it is much better, simpler and more consistent to define feature flags (options) in profile files. You define them in those profile files, and then everybody on the dependency graph agrees on the features, it is possible to build liba/Qt independently on the CI and the binaries created will satisfy the consumers that later will require that package.

As a summary, we don't want to go back to the options model in Conan 1 that had the behavior that you expected from your suggestion of the "feature-flags" feature, because it is both problematic from the point of view of conflicts and from the distributed and binary management point of view.

The current options model together with the definition of the options values in the profile files, instead of distributed in the recipes in the dependency graph, would be the recommended approach for feature flags implementation.

Thanks very much for the feedback!

[IsaacDynamo]

Hi @memsharded,

Thanks for your in-depth answer and Conan history.

I agree that the additive feature flags model doesn't work for cases that are not additive, like disabling functionality. And for that use-case global configuration via a profiel as a single source of through is a nice way to sidestep conflicts.

But as far as I understand options management via profiles has to be done manually. And not only for direct dependencies but also for transitive dependency. This is different from version constraints which are only placed on direct dependencies, but Conan is still able to resolve versions for the whole dependency tree.

Additive feature flags could be seen as additional constraints, "give me a configuration with at least these features enabled". Which can be resolved automatically and conflict free via the union of all requested features. These feature flags would be more restrictive then the current options because there behavior has to be additive, so things like without_xx are not allowed, because that would remove functionality. But the additive restriction would enabled automatic feature configuration over the whole dependency tree, just like version resolution.

The automatic feature configuration has to be done on the whole tree, and this information could then be used in independent pipelines. Just like a profile, that also has to be determined before hand for the whole tree.

So I still think additive feature flags would be a nice addition to Conan, not to replace the current options, but enable automatic feature configuration for purely additive features.

But given the history and low interest, I can understand that it is unlikely that such a system will ever be added.

---

So I will follow your recommendation and use the current options model together with a profile.

But I'm still a bit unclear how the options configuration for the profile is determined. For direct dependency the used features are known, but for for transitive dependencies the used features are an implementation detail of the direct dependencies.

So how is this profile configuration typically done? It feels like I'm missing something obvious.

To give an example, if

• A depends on B and C
• B depends on D with feature F1
• C depends on D with feature F2
• D has no dependencies

How do I discover that D should have features F1 and F2 enabled? Because it is unclear where the "with feature F1" is even specified/documented if self.requires("D", options={"with_F1": True}) is no longer recommended.

[memsharded]

But as far as I understand options management via profiles has to be done manually. And not only for direct dependencies but also for transitive dependency. This is different from version constraints which are only placed on direct dependencies, but Conan is still able to resolve versions for the whole dependency tree.

That is not totally accurate. Version resolution can also need profile definitions, for example [platform_requires] [replace_requires], [tool_requires], and it might also need to resolve conflicts in downstream consumer recipes via override=True or force=True traits, explicitly. Version ranges are not resolved in a full joint-compatibility manner, because that doesn't escale (see explanation below)

Additive feature flags could be seen as additional constraints, "give me a configuration with at least these features enabled". Which can be resolved automatically and conflict free via the union of all requested features. These feature flags would be more restrictive then the current options because there behavior has to be additive, so things like without_xx are not allowed, because that would remove functionality. But the additive restriction would enabled automatic feature configuration over the whole dependency tree, just like version resolution.

Please note that this has another undesirable side effect, it requires some back-tracking. With the constraint of "additive-only", it doesn't really becomes NP (as a full search of compatible versions over version ranges would be), but it can still increase a lot the necessary processing time of dependency graphs. Because everytime a feature flag changes, practically the full graph has to be re-evaluated. Because recipes have conditional requirements based on options/feature-flags, but also they change how they affect their consumers (e.g. a feature flag enables a compiled library for a case it was previously a header-only), so they need re-evaluation of the consumers, which in turn can decide to use a different version. When graph is invalidated and new versions are requested they need to be downloaded from the server, etc. The processing time from an average graph can go from <1 minute, to a few minutes, which in terms of UX is a lot, it basically breaks the developer flow and convenience of the tool.

So I still think additive feature flags would be a nice addition to Conan, not to replace the current options, but enable automatic feature configuration for purely additive features.

But given the history and low interest, I can understand that it is unlikely that such a system will ever be added.

The CI case would still be broken if feature flags were implemented, and this is an important and big case to be disregarded.

So yes, overall I think this cannot be added, but not specifically because of history and low interest, but because of the technical and UX challenges that would derive from this feature that are huge compared with the value proposition.

---

How do I discover that D should have features F1 and F2 enabled? Because it is unclear where the "with feature F1" is even specified/documented if self.requires("D", options={"with_F1": True}) is no longer recommended.

A possible approach is to use the validate() method of the consumer, and raise an error if the specific feature of the dependency is not enabled. You can of course still use self.requires("D", options={"with_F1": True}), the only caution is to be aware that this is not always enforced and it can have other values.

So the recommendation would be:

• Try to make the D default to have with_F1 and with_F2 active by default. This is the simplest approach.
• You can protect and request features in the consumers B and C validate() methods to check the self.dependencies["pkgd"].options.with_f1, for example
• When the consumers raise an error, they will clearly inform: "package C cannot use package D because it needs F1 enabled, and it is not, please enable it in your profile"
• When this is done the actual UX is pretty good. It takes just a few seconds to add the necessary things to the profile (in case D didn't have those defaults already), and the application of that profile everywhere will make a solid experience, making sure that D is built with the right features even when built in isolation, that B and C will be built against a D binary that has F1 and F2 available (depending on the features, it can be important for the binary compatibility of B and C), even when they are build in isolation, and all the binaries will be aligned and reduce a ton potential binary compatibility issues that otherwise happen later, often in runtime.

[IsaacDynamo]

Thanks for the additional explanation and recommendation! It clarified a lot.

I will give options an other shot, this time with validate() and profiles.