rust-lang / cargo

The Rust package manager
https://doc.rust-lang.org/cargo
Apache License 2.0
12.74k stars 2.42k forks source link

Platform target-specific features #1197

Open alexcrichton opened 9 years ago

alexcrichton commented 9 years ago

It would be nice to support target-specific features for cases such as when certain functionality is only available on one platform or the default set of functionality should vary per-platform.

alexcrichton commented 9 years ago

cc @huonw

nox commented 8 years ago

Shouldn't this work?

[target.'cfg(not(any(target_os = "android", target_os = "windows")))'.dependencies]
ipc-channel = {git = "https://github.com/servo/ipc-channel"}

[target.'cfg(any(target_os = "android", target_os = "windows"))'.dependencies]
ipc-channel = {git = "https://github.com/servo/ipc-channel", features = ["inprocess"]}
alexcrichton commented 8 years ago

@nox in theory yeah that seems like it should work, but today due to the structure of Resolve it doesn't. The entire target-independent resolution graph is created and then only later filtered down to the relevant set of packages. As an artifact of creating Resolve at the beginning it resolves all features, basically with a map from package to list of features. That means that all instances of ipc-channel in your example would have the inprocess feature enabled, regardless of what platform it was on.

I think that's a fixable bug, however, although likely quite invasive as it would require deferring calculation of features to the compilation phase rather than the resolution phase.

jethrogb commented 7 years ago

Ran into this extremely confusing bug today...

gyscos commented 7 years ago

Would this also allow specifying a target-specific features.default array? Something like:

[target.'cfg(not(target_os = "windows"))'.features]
default = ["general_feature"]

[target.'cfg(target_os = "windows")'.features]
default = ["winonly_replacement"]

I suppose a workaround to get this would be an intermediate crate which depends on the actual crate with target-dependent features, but it doesn't sound very convenient.

alexcrichton commented 7 years ago

@gyscos that seems like a nifty and natural way to encode this information! I haven't though too much about the implications of target-specific features, but I don't particularly see any immediate reason to block anything.

nox commented 7 years ago

This isn't enough, I don't want my crate to stop working because someone used no-default-features.

I guess one could use @retep998's trick of enabling a feature programatically from a build.rs script.

retep998 commented 7 years ago

@nox That enables features in code programmatically. It doesn't let me control dependencies, but for some people just controlling their code might be enough. The reason I even wrote that trick was due to the lack of support for cyclic features. https://github.com/retep998/winapi-rs/blob/dev/build.rs#L193

philn commented 7 years ago

What's the status of this issue? It would be interesting to use target-specific features in gecko-media (enable pulseaudio only for linux for instance).

alexcrichton commented 7 years ago

@philn AFAIK no progress has been made

praetp commented 6 years ago

Hope we can see progress on this issue...

huangjj27 commented 6 years ago

This will help a lot with https://github.com/rust-lang-nursery/rand/issues/442

plyhun commented 6 years ago

So is this a bug ( the [target.'cfg(target_os = "android")'.dependencies] / target.'cfg(target_os = "android")'.features notation should work, but it does not ) or a feature that requires a RFC?

alexcrichton commented 6 years ago

@plyhun it's intended behavior that it's not supported, but it's a bug if Cargo doesn't warn you about unused keys. I'm not sure about whether or not adding the functionality requires an RFC, but if it becomes semi-large than I think it might!

stefandeml commented 5 years ago

is there any work-around for this besides creating an intermediate crate which depends on the actual crate? - that's very ugly. Especially for code targeting wasm this could be very helpful. Thanks

chris-morgan commented 5 years ago

Making [target.'cfg(…)'.**] work for everything, rather than just [dependencies], seems to me like it should be a straightforward and sensible change to make. I can’t see any downsides to it. Am I missing anything?

MOZGIII commented 5 years ago

I naively expected [target.'cfg(…)'.**] to work for everything. I guess I'd be great to if this was the reality - after if I expected it other people might've too.

That said, it might make sense to enable it just for the features - it's the safer way, because we might not want to allow that for [package] ([target.'cfg(…)'.package]) for example.

My use case is to write something like this:

[dependencies.tui]
version = "..."
default-features = false
optional = true

[dependencies.termion]
version = "..."
optional = true

[dependencies.crossterm]
version = "..."
optional = true

[features]
crossterm_backend = ["crossterm", "tui", "tui/crossterm"]
termion_backend = ["termion", "tui", "tui/termion"]

[target.'cfg(windows)'.features]
default = ["crossterm_backend"]

[target.'cfg(not(windows))'.features]
default = ["termion_backend"]

I don't think this can currently be achieved, unfortunately. :( The tricky part is passing the features to dependency conditionally. Is there some way I can do it today?

Minoru commented 5 years ago

Can someone please explain how to work around this with a dummy crate? I tried, but I always end up with the feature enabled. My question on the forum has all the details of what I did, but no-one answered it yet.

aknuds1 commented 5 years ago

Just ran into the bug that target specific dependency features are applied also for other targets :( Is there any chance of getting this bug fixed?

ehuss commented 4 years ago

Target-specific features for dependencies has been implemented and is available as a nightly-only feature on the latest nightly 2020-02-23. See the tracking issue at #7914.

If people following this issue could try it out, and leave your feedback on the tracking issue (#7914), I would appreciate it. Particularly we'd like to know if it helps your project and does it cause any breakage?

I'm going to leave this issue open, for the request of also changing the default features.

silverweed commented 3 years ago

Hello, what's the status of this issue? Is there still hope for it to see the light of day (and stabilization)?

JAicewizard commented 3 years ago

features for dependencies has been implemented and is available as a nightly-only feature on the latest nightly 2020-02-23. See the

From what I can tell that only implements target specific dependencies, not target specific features. So:

[target.'cfg(target_os="linux")'.features]
default-backend = ["gtk"]

[target.'cfg(target_os="macos")'.features]
default-backend = ["mac"]

[target.'cfg(target_os="windows")'.features]
default-backend = ["win"]

Still doesnt work while I would expect it to work given that this this PR is supposed to solve this (github) issue.

JAicewizard commented 3 years ago

I got a working draft of this, its currently one big commit over at https://github.com/JAicewizard/cargo/tree/target_specific_features.

@ehuss could you help me with getting this into cargo? There is probably an RFC process, but I think this issue shows good support for this already. I think its more important to the code into a state where it can be put into cargo. Feature flags, stabilisation process etc.

Code is definitely not clean ATM, I plan on splitting it up into different commits tomorrow.

ehuss commented 3 years ago

@JAicewizard I think it would be good to start by writing some clear motivations and use cases. Alex can't remember why he opened this issue, and we couldn't think of any use cases that aren't already solvable by existing means.

drahnr commented 3 years ago

The prime use case I would have is, to have a feature that is not-mandatory for functionality, but it only works on certain OSes, i.e. jemalloc-ctl which is only viable on unix platforms when used with jemalloc, where it should be enabled by default, but not on windows. So that'd require target dependent default features or a similar mechanism. The approach as shown in https://github.com/rust-lang/cargo/issues/1197#issuecomment-897501839 does not work with 1.54.0 stable.

C0D3-M4513R commented 3 years ago

I am running into the same issue. I am currently writing a dll injector, as a library. I plan to implement it on all platforms. On windows I need some windows specific crates. On Linux I'll need other crates. The way, things are currently, I'd just end up compiling all libraries, if I don't wanna implement target-specific features for every os.

I do not wanna add target-specific features, the user has to supply, because my goal is, to make the library, that I am writing, completely platform independent, to the one using it. We have target specific modules in rust, but we do not have any way, to define dependencies, for those target specific modules.

Ralith commented 3 years ago

When building a cross-platform application, sometimes I want FFI bindings to download/build/statically link the foreign library automatically on Windows, but use the system package on Linux, where installing stuff doesn't suck as bad. This often maps to wanting to enable a feature to bundle the foreign library on Windows only.

JAicewizard commented 3 years ago

My use case is for druid which is a GUI toolkit. My design for backend selection relies on features, and we need one (or more) default backend(s) for each target OS that are compiled into the binary. This would allow someone to use a GTK backend on windows or mac for example, same for X11 (using some X11 compatibility layer). This would mostly just be for experimentation, but it doesnt have to be.

Even outside that, it would allow more clean cargo.tomls + code communication. You dont need complex cargo.tomls with lots of targets each with 5 features, instead you have 1 feature for each target each including their optional dependencies. Then the existance of these dependencies can be easily checked by locking it behind the feature that enables these dependencies. This is much cleaner and allows for easier feature management.

Ericson2314 commented 3 years ago

I think the current syntax makes this more confusing that it needs to be. The current [features] section does two things: it declare what the features are, and specifies the dependencies between features.

I think based on what the others say there is good need for target-specific dependencies. Note that this includes target specific default features as they are target-specific dependencies of the "default" feature.

However, there is no need for the mere existence of features to vary between platforms. It would be much better to solve that with something like Cabal's "buildable: false". Not only can that nicely express not all features work everywhere, it also can be used to express that certain features are incompatible wihout violating the additivity of features! (Technically, one can express "buildable: false" today with an impossible depenency, so we know it's safe. This just takes that existing functionality and makes it more ergonomic.)

This, and weak deps as I described in https://github.com/rust-lang/rfcs/pull/3143, are both cases of things we can't get today, but a more flexible language for specifying dependencies would get is for free. I really hope we can get that more flexible language!

CC @Eh2406

drahnr commented 3 years ago

I think the current syntax makes this more confusing that it needs to be. The current [features] section does two things: it declare what the features are, and specifies the dependencies between features.

I disagree, it is in line with the target specific dependency declaration so imho it's consistent.

I think based on what the others say there is good need for target-specific dependencies. Note that this includes target specific default features as they are target-specific dependencies of the "default" feature.

However, there is no need for the mere existence of features to vary between platforms. It would be much better to solve that with something like Cabal's "buildable: false". Not only can that nicely express not all features work everywhere, it also can be used to express that certain features are incompatible wihout violating the additivity of features! (Technically, one can express "buildable: false" today with an impossible dependency, so we know it's safe. This just takes that existing functionality and makes it more ergonomic.)

Good catch! Couldn't this be solved by an impl detail to check if a feature exists at all before checking if it exists for the current target, and show a sufficiently detailed error message? Iiuc that would be equal to an explicit buildable flag. Could you reference where this is outlined? Thanks!

JAicewizard commented 3 years ago

However, there is no need for the mere existence of features to vary between platforms

I dont think that is true. We can already request optional dependencies (which are features) form another platform to be included, and this works on stable. I think my implementation also has a similar behaviour to these dependencies. On targets where the features dont exist, they are just ignored/are empty.

It would be much better to solve that with something like Cabal's "buildable: false".

That would require the users to select the features, I want to give the user one stable API on diferent platforms without the users having to ask for a diferent feature on each platform.

Ericson2314 commented 3 years ago

@JAicewizard

We can already request optional dependencies (which are features) form another platform to be included, and this works on stable.

Requesting conditionally is perfectly fine. It's providing features optionally to downstream creates conditionally that's suspicious.

That would require the users to select the features, I want to give the user one stable API on diferent platforms without the users having to ask for a different feature on each platform.

That is what the conditional default features do. Isn't that what you want?

@drahnr

I disagree, it is in line with the target specific dependency declaration so imho it's consistent.

[target."....".features] is consistent with [target."....".dependencies], but only the former has the issue I mention I believe? It's precisely the fact that doing the consistent runs into issues that means we should perhaps back up and fine something more different ("a more flexible language for specifying dependencies") so we can be consistent and have good semantics.

Good catch! Couldn't this be solved by an impl detail to check if a feature exists at all before checking if it exists for the current target, and show a sufficiently detailed error message? Iiuc that would be equal to an explicit buildable flag.

That somewhat works, but i don't think is as clean. What we want to express is "Foo depends on macOS or Linux", not "If mac or linux exists there is a Foo". I think the framing really matters as to what users end up writing and how they feel about it.

Could you reference where this is outlined? Thanks!

https://cabal.readthedocs.io/en/3.4/cabal-package.html?highlight=buildable%3A#pkg-field-buildable. The docs mention it in conjunction with an autoconf-ish setup, but it is more commonly used these days just with regular conditional deps, such as in https://github.com/reflex-frp/reflex-dom/blob/master/reflex-dom/reflex-dom.cabal (which incidentally I think is pretty close to @JAicewizard's use-case).

JAicewizard commented 3 years ago

@Ericson2314

Requesting conditionally is perfectly fine. It's providing features optionally to downstream creates conditionally that's suspicious.

I dont think you understand what I mean. Currently we can already provide features that are target specific.

[target."cfg(target_os=linux)".dependencies]
dep1 = {version = x, optional = true}
Ericson2314 commented 3 years ago

@JAicewizard can a downstream create refer to dep1? I thought dependencies overloaded features as if they were "private" features, since the crate can refer to its own optional dependencies but not a downstream crate.

I agree in that this is no good, but if it doesn't like into the crate's public (cargo-side) interface, then it is at least local and could be fixed with an edition.

JAicewizard commented 3 years ago

@Ericson2314 Yes, if you are on a target with no explicitly listed dependency with that name it will just ignore it. At least from my testing.

Ericson2314 commented 3 years ago

@JAicewizard say that again? In downstream crates I would expect trying to refer to any dependency in an upstream crate (that doesn't share a name with a feature in which case we are referring to the feature) to be an error, not silently ignored.

I take it you mean within the same crate declaring the target-specific dependency?

JAicewizard commented 3 years ago

@Ericson2314 https://doc.rust-lang.org/cargo/reference/features.html#optional-dependencies

Cu3PO42 commented 3 years ago

I believe I have a use-case for this. I am working on a cross-platform tool that needs to interact with a USB device. I use libusb via rusb to achieve the majority of the abstraction. On Windows, I need a special, patched version of libusb to get around a limitation. I therefore enable the vendored feature on the libusb1-sys crate, which builds and statically links libusb.

My dependencies are as follows:

[workspace]
[package]
# ...
edition = "2018"

[dependencies]
rusb = "0.8.1"
# ...

[target.'cfg(windows)'.dependencies]
libc = "0.2"
windows = "0.17.2"
win32_bindings = { path = "src/backend/windows/win32_bindings" }
libusb_internals = { path = "src/backend/windows/libusb_internals" 

where libusb_internals itself has the following dependencies:

[dependencies]
libusb1-sys = { version = "0.5.0", features = ["vendored"] }
win32_bindings = { path = "../win32_bindings" }

[build-dependencies]
cc = "1.0"

On other platforms, I would rather dynamically link libusb, however, the "vendored" feature appears to be picked up and libusb is being built. I also see libusb_interals-* folders in target/debug/build/ which makes sense given that target-specific dependencies are only filtered later in the process. All this wouldn't be a huge problem if it weren't for the fact that libusb1-sys apparently doesn't set up the correct environment on (older versions of) macOS and fails to build libusb. I can manually build libusb or install it from Homebrew without problem, though. Manually commenting out the Windows-specific libusb_internals dependency lets my project build correctly on macOS.

This leaves me at an impasse where it seems impossible to configure my project correctly for multiple target platforms.

kaimast commented 2 years ago

Somewhat related: Cargo.lock currently tracks dependencies/features across all build targets which can be a problem.

Here's an example: I have an codebase I build for WebAssembly and as a native binary. In the latter case I use the rand crate with std and getrandom. However, getrandom does not support WebAssembly so it won't compile, even when I explicitly disable all default features on the wasm target.

CryZe commented 2 years ago

@kaimast Are you using Rust 2021 and / or resolver 2? This should not be happening if either of those are the case. (Unless of course a crate legitimately pulled in by WASM does indeed activate that feature)

Although wait, what do you even mean? getrandom does support WASM, you actually need to activate a feature for it to work, not vice versa.

federico-terzi commented 2 years ago

If anyone is still searching, I've been able to work around this problem by using a combination of features and a cargo-make script: https://github.com/espanso/espanso/pull/1287/files

My use-case was very similar to the one by @Cu3PO42, as I wanted to enable/disable certain reqwest features based on the OS

IceTDrinker commented 1 year ago

What’s the status on this?

Where should someone look to try and make this work?

Have there been PRs trying to do something about it?

Cheers

JAicewizard commented 1 year ago

See https://github.com/rust-lang/cargo/issues/1197#issuecomment-897501839 for more information, basically the team forgot why this would be useful and didn't really have any intentions of pursuing this any further. But I have a probably working branch implementing this, just for an older version of cargo.

Nukesor commented 1 year ago

Would love to have this as well! I actually have a pretty good usecase right now.

I'm trying to create a shared lib crate used by

To enable the FFI logic, we utilize a code generator crate. When using a target specific dependency via [target.armv7-linux-androideabi.dependencies] to include this crate, the whole dependency graph of that dependency is also included in the esp related project. This then messes things up resulting in a compilation error, as the project's resolved dependency versions change due to this new dependency. I suspect that some std features are included, which don't work on the esp32 toolchain.

Long story short, the only thing that worked was hiding those FFI crates behind a feature flag.

It would be great if one would be now enable to enable feature flag based on that specific target.

Canop commented 1 year ago

If what blocks progress here is the lack of use cases, here's mine.

broot has a clipboard feature, which works on some targets and doesn't on some other ones. At various point in the code, I must check whether the feature is available. Checking both the feature and the combination of targets, and updating those checks everywhere when there's progress in the feature implementation on various targets would be a mess. Right now, users can't do a simple cargo install, they must, depending on their platform, add --features clipboard themselves.

fogti commented 1 year ago

@Canop couldn't that be handled by doing this target-to-feature inference once in build.rs? Obvious downside would be that it is not as declarative (but I'd assumed this feature to be more about cases where a feature wouldn't be reasonably possible to implement, e.g. io_uring support on something that isn't linux, or such).

Canop commented 1 year ago

@fogti You mean by outputting cargo:rustc-cfg=clipboard in build.rs ? That's honestly something I didn't think about (and didn't know possible, thanks). This might work but the fact that Cargo conditional dependencies aren't affected makes it messy again (but at least not everywhere in the code).

DusterTheFirst commented 1 year ago

@fogti You mean by outputting cargo:rustc-cfg=clipboard in build.rs ? That's honestly something I didn't think about (and didn't know possible, thanks). This might work but the fact that Cargo conditional dependencies aren't affected makes it messy again (but at least not everywhere in the code).

I think you could also check if the feature is enabled in the build script and error out or give a warning if the feature should be enabled

terakilobyte commented 1 year ago

Adding another use-case. One of our dependencies has a feature that links against a specific library that we'd like to enable for a specific os. Currently, we're passing --features crate/feature on that one specific OS, but it'd be nice to handle this in Cargo.toml

glandium commented 1 year ago

AFAIK, the syntax in https://github.com/rust-lang/cargo/issues/1197#issuecomment-231584705 works currently, but the one that is still lacking is the one from https://github.com/rust-lang/cargo/issues/1197#issuecomment-268203727.