search

rust-lang / cargo

The Rust package manager
https://doc.rust-lang.org/cargo
Apache License 2.0
12.63k stars 2.4k forks source link

Ability to disable individual default features #3126

Open ideasman42 opened 8 years ago

ideasman42 commented 8 years ago

The current documentation here: http://doc.crates.io/manifest.html states:

With the exception of the default feature, all features are opt-in. To opt out of the default feature, use default-features = false and cherry-pick individual features.

This is quite rigid, since it means you can't make a small adjustment to defaults - where a program may have multiple options which aren't related.

It also means if I use a 3rd party package with modified defaults, after an update I need to check if defaults where added which I might want to enable.

Note, I asked this question on stackoverflow, it was suggested to open an issue here: http://stackoverflow.com/questions/39734321

alexcrichton commented 8 years ago

Yeah it's true that default features are a very weighty decision and are quite difficult to reverse. I don't think we can add the ability to force-disable them, however, because how would we know whether a dependency actually needs the feature or not?

ideasman42 commented 8 years ago

@alexcrichton, maybe this is a bigger change then I'd expected, am not very experienced using Cargo.

Could dependencies list features they require?

alexcrichton commented 8 years ago

Dependencies already do, the default feature is just special where it's turned on by default. If a crate has a feature that may be disabled then in general crates shouldn't move it to a default feature.

ideasman42 commented 8 years ago

In that case wouldn't it be possible to disable a default - having the same behavior as if you explicitly listed all defaults, without the one(s) which have been requested to be disabled?

alexcrichton commented 8 years ago

Yeah you can disable default features with default-features = false, but Cargo also unions features requested for a crate so if any crate doesn't disable a default feature then it ends up enabled.

tbu- commented 7 years ago

This is a backward-compatiblity hazard. E.g.: It seems that libcore is getting default features, relatively harmless ones, they change some formatting. Some crate today might opt out of default features and only select the float formatting thing. This means that libcore can never add another default feature that disables existing stuff without breaking that crate.

The better way to handle that would be that each crate has a list of default features that it does not require. This way, default features can be added in the future.

ideasman42 commented 7 years ago

I don't know about libcore but at an application level - it often makes sense to be able to disable dependencies (which are default since they are used in official release builds).

In for you may want to build FFMPEG with patented codecs or not. You may want to build a video editor with FFMPEG or not. This is typically the case for file formats, codecs, or optional scripting languages (as VIM has with Python, Ruby, Lua... etc)

tbu- commented 7 years ago

Note that I don't talk about hard-disabling dependency feature, but rather the equivalent of today's default-features = false and then a list of all the other features in the features = [] list. That means if some other dependency pulls in my disabled default feature, I still get it.

alexcrichton commented 7 years ago

@tbu- the idea seems sound to me at least!

kornelski commented 6 years ago

I'd especially like this to work on command line. --no-default-features is lengthy, and requires user to repeat other defaults. A syntax sugar for it would be very helpful.

Here's a proposal:

Features prefixed with - are removed from the set of default features, i.e.:

D = default features F = user-specified features without - prefix N = user-specified features with - prefix

Currently:

features = D ∪ F

Proposed:

features = (D ∖ N) ∪ F

e.g. given 

[features]
default = ["foo", "bar"]
foo = []
bar = []
quz = []

--features=-bar,quz is desuraged to --no-default-features --features=foo,quz

CAD97 commented 2 years ago

This would also be very useful for testing e.g. --all-features --features -legacy-compat.

(Though ofc you do need to worry about specifying the interaction with feature dependencies.)

epage commented 2 years ago

Going to do a quick brain dump since this has been on my mind...

When it comes to evolving features without a breaking change, the biggest hazard is taking functionality that already existed and making a new feature from it. Existing features can already be split if you are ok with not reusing existing names though deprecation support would help a lot.

In cargo's existing model of default-features=false, existing functionality that gets split into a feature is being split out of the "base" functionality. https://github.com/rust-lang/rfcs/pull/3283 works to solve this by adding an explicit "base" concept with a no-default-features feature that you can add to.

If we had started from scratch inventing default features, another approach to breaking existing functionality into a feature would be if we allow subtracting features from default. https://github.com/rust-lang/rfcs/pull/3146 called this out as an alternative but it has the following challenges

I think limiting the scope of feature opt-outs and ensuring design keeps the intent clear can overcome these challenges. I also think we can transition to this even if we didn't start with this.

A rough outline of my proposal

It would be interesting to see if this would help with --all-features but I would not see that as blocking on the design.

Of course, there are details to be worked out and bike shedding to be done.

Other benefits

Open questions

CAD97 commented 2 years ago

If we allowed general feature opt-out, it could likely break crates as they might check for a feature and assume its required features are available.

FWIW I would expect -foo to disable any features that have foo as a requirement (but other features that were activated because of it to stay on).

Explicit non-goal to force/assert the deactivation of features

Yes, it's removing the requirement, not the feature itself.

I agree that opting out of future default-features is unfortunate. But rather than move to a default feature, it seems like we could "just" allow opting out of default-features's specified features, and discourage/deprecate setting no-default-features for a manifest dependency.

A default feature then can be just a pattern for allowing opting out of a chunk in one opt-out like today's no-default-features, rather than cargo-recognized.

epage commented 2 years ago

FWIW I would expect -foo to disable any features that have foo as a requirement (but other features that were activated because of it to stay on).

For this to work, it could only have that affect within the current crate's resolving of features. I worry these semantics are too close in semantics to asserting that a feature won't be present at all (remove requirement everywhere) that it could be confusing.

I also think it has the chance to break people. I'm going to expand on a case in RFC 3146. Say I have a dependency with

[features]
default = ["foo", "bar"]
foo = []
bar = []

and I depend on it with ["foo", "-bar"]

And later the dependency changes it to

[features]
default = ["foo", "bar"]
foo = []
bar = ["foo"]

Then according to these semantics, foo will be disabled on upgrade, breaking dependent crates. This is why the earlier proposal was looking to special case default's semantics into what I'm terming a meta-feature so only default will be in a quasi-state because we only modify its requirements but no one will be allowed to observe that quasi-state, making it work out.

CAD97 commented 2 years ago

Then according to these semantics, foo will be disabled on upgrade

not by my intuitive understanding, though explaining it is a bit difficult. -bar would disable bar and default, but not foo, even if foo was not explicitly enabled.

.... also wait we set cfg(feature = "default") I did not know nor expect that

CAD97 commented 2 years ago

explaining it

epage commented 2 years ago

.... also wait we set cfg(feature = "default") I did not know nor expect that

I assumed we didn't either but I went and tested it. We only set it if the user explicitly mentions a default feature

epage commented 2 years ago

If any feature removed in the previous was explicitly requested, error

Wouldn't this still be a semver breakage for bar to require foo? Without the error, we are disabling foo when code was written assuming it was enabled. With an error, we went from a working build to a failing build on cargo update since foo will have been removed because bar was removed and foo was explicitly requested.

epage commented 2 years ago

At RustConf, one of the problems that was pointed out is that default-features = false would be removed on an edition boundary of the dependent so a dependency cannot guarantee when all of their dependents are on the new edition to start relying on the new semver semantics.

Quick thoughts

Nemo157 commented 1 year ago

One thing that seems not possible with the proposed design is adding a new default-active std-using feature to an optionally std-using library.

Imagine we have a crate foo which has features:

[features]
default = ["std", "bar"]
std = ["dep:std"] # with sysroot dependencies, otherwise `#[cfg(feature = "std")] extern crate std;`
bar = []

In a no-std project we then depend on this crate and disable the std feature:

[dependencies]
foo.version = "1.0.0"
foo.features = ["-std"]

Now, foo wants to introduce a new feature baz which requires std to implement, it also wants this to be provided by default:

[features]
default = ["std", "bar", "baz"]
std = ["dep:std"]
bar = []
baz = ["std"]

The downstream project on a non-breaking update of foo would now have the std dependency activated, transitively through the new default-feature baz. This would break any build on a target which does not have std available.

djc commented 1 year ago
  • When depending on something declaring, say 2024 edition, the dependent can't set default-features = false

The issue with this is that the dependent can no longer update to 2024 edition without doing a semver-breaking release, right?

I've thought for a long time that the default features design including the required additivity of features is problematic. The problem here is that any crate in your dependency graph can enable a feature in a crate you depend on without you being aware. Being able to specifically request feature disabling would make a lot of this stuff easier, I think, if Cargo would guarantee that a crate would fail to build if some part of the dependency graph tries to enable a feature that I've requested be disabled.

While that's a decent departure from the current state of things, I feel it is more aligned with the Rust philosophy of builds failing early if you don't have your types lined up correctly: you can specify that you don't want some feature and if there's some "spooky action at a distance" that's enabling that feature anyway, Cargo will tell you about it.

The downstream project on a non-breaking update of foo would now have the std dependency activated, transitively through the new default-feature baz. This would break any build on a target which does not have std available.

Maybe the solution here would be extending the notion of weak dependencies to features, so it becomes expressible that a feature should only be part of the default if std is enabled? Straw man: default = ["std", "bar", "std? ( baz )"]. As I argued in the aforelinked internals post and also here, I think there could be a lot of value in having a richer way to express features and dependencies. (For more info, Gentoo's development manual has the complete syntax.)

epage commented 1 year ago

One thing that seems not possible with the proposed design is adding a new default-active std-using feature to an optionally std-using library.

Correct, this is not intending to cover every form of feature evolution but is focused on splitting built-in behavior into default features.

Nemo157 commented 1 year ago

So, the intent is not to just enable new kinds of feature evolutions, but rather change the set of allowed feature evolutions, removing ones that are assumed to be unnecessary like my example (which is possible with todays default-features = false formulation) in favor of ones like moving ungated functionality into features?

epage commented 1 year ago

Good point that this might be adding a new way of things breaking; I've added a note to my proposal that that needs to be looked into.

djc commented 1 year ago

IMO it would be a major constraint that adding new features that require std would need a semver-breaking release.

kornelski commented 1 year ago

So perhaps specifying "not foo" should mean "do not enable any default feature of this crate that results in foo being enabled", so if there's default = ["bar"]; bar = ["foo"], it disables bar as well.

tmccombs commented 1 year ago

The issue with this is that the dependent can no longer update to 2024 edition without doing a semver-breaking release, right?

One way to solve this could be to create a new special feature with a different name, let's call it new-defaults. If a crate has new-defaults defined, then it can't be opted out of with default-features = false, and using the old default mechanism can be deprecated in a new edition.

Ericson2314 commented 1 year ago

A lot of people say they want negative features, but I find the idea hard to discuss because there is usually much hand-waving about what the semantics actually are. The current additive semantics are deeply part Cargo; we see the same semilattice stuff both the intersecting of version bounds and unioning of feature sets, in fact.

There seems to be two variants, I will call them "hard" and "soft"

"hard" means "no definitely do not do this" as @kornelski I think says. @epage says in https://github.com/rust-lang/cargo/issues/3126#issuecomment-1190950930 that this is out of scope and I think that is the correct decision. This sort of thing is deeply non-compositional. The first idea of package management is that we can always "add more stuff", indeed thing likes the "orphan rules" go out of their way to restrict crates so this is preserved. People will abuse the "hard" want to merely mean "I don't need this", and then we'll have endless issues with people needing to fork crates to remove artificial restrictions. A disaster.

"soft" means "I don't need this", and is what the idea from @epage is. This is compositional, because it is just sugar: the underlying model is still additive. but I think it is also just confusing. Feature dependencies can easily bring back an opted-out feature. The combination of multiple crates' depedencies, or even multiple features dependencies from the same crate as @Nemo157 points can also do the same. People will misunderstand the "soft" as the "hard", and get even more confused about how Cargo actually works.

Another thing to think about is

If this new version of the library was actually the only version, would one use soft negative features?

My guess is the answer is "no". Soft negative features, because their confusing UI, are only worth it as crude feature migrations. So over time we get a mix of positive and negative features which don't make sense in isolation, but only as an accident of history. This I think is a bad user experience, and the classic pitfull of borrowing from future users to pay of today's debt.

The only solution that allows users to both avoid compatibility issues and write issues that make sense "in the moment" and not compromises with history is some extra indirection notion of feature migrations / feature name-spacing / etc.. So I think our goal to be to get there eventually.

Recall how it is proposed that root crates should be able to violate the orphan rules. I think similarly it does make sense for root crates / virtual workspace roots to be able to have hard negative features. That satisfies the "help I need to disable this thing" problem while avoiding the huge issues of compositionality. Yay!

tmccombs commented 1 year ago

I think similarly it does make sense for root crates / virtual workspace roots to be able to have hard negative features.

That would introduce a backwards compatibility hazard. If Project A forbids feature X in library B, and also depends on library C, then if C adds a dependency on Feature X from B, it will break project A.

And by itself, it still doesn't solve the problem of splitting out a new feature from the base. Because doing so would still break libraries that use default-features = false.

Ericson2314 commented 1 year ago

@tmccombs I don't think compatibility of that sort matters with the workspace root. If you really don't want feature X, then you don't want any version of C that requires X.

C shouldn't feel boxed in because now (unlike before), users can forbid X, but anyone that forbids X knows the penality may be some build plans are ruled out --- just as they asked for!

Basically we have to always step back and ask what we are trying to solve. For intermediate noes in the the dependency graph the "mechanism design" is very complicated, we want Cargo.tomls to express concepts that are compositional and maintain the health of the ecosystem as whole. But for concrete projects not reusable components we don't need to have such high-minded concerns: let them do whatever they want and alone (no contagion) suffer the consequences.

djc commented 6 months ago

I just ran into this again: rustls (and, downstream of that, tokio-rustls) switched the default crypto provider in their latest releases (0.23 and 0.26 respectively), with the new releases allowing downstream users to switch crypto provider dependencies via Cargo features. I upgraded tokio-rustls and used default-features = false, features = ["ring"] to go back to the "old" default. However, this in turn also disabled other default features, including tls12, which might have caused our proxy endpoint to become harder to reach for many kinds of HTTP clients. Would be really nice to have this feature.

epage commented 6 months ago

I think there is solid interest in this. The problem is availability for someone to go through the design process. For myself, I already have about a years worth of design work I've committed to.

(this is in a similar boat to mutually exclusive, global features)

djc commented 6 months ago

So like, start with a pre-RFC on internals and work from there?

epage commented 6 months ago

Yes, though first making sure to read over this thread so any nuance in it gets captured.

tbu- commented 6 months ago

I don't think a third-party solution will be accepted by the Cargo team.

My two-year old https://github.com/rust-lang/rfcs/pull/3283 would have solved it, in a way that other people are already able to hack around Cargo's lack of proper support for default-features: https://slint.dev/blog/rust-adding-default-cargo-feature. It was closed for being too complex, but I think it's the minimum complexity to make such a feature work at all. Another approach at https://github.com/rust-lang/rfcs/pull/3347 has stalled for 1.5 years.

kornelski commented 6 months ago

To make it clearer to users that this doesn't disable features globally I suggest a couple of mitigations:

  1. Give it a verbose descriptive syntax, such as "default-without:foo".

  2. Warn when these features end up enabled anyway. Now that there's [lints] it can be used to acknowledge it if necessary.

narodnik commented 1 month ago

ITT: bike-shedding