Cargo standard library dependencies

[ehuss]

This issue is for tracking the writing of an RFC for extending Cargo.toml to specify dependencies on standard library crates.

There are a few needs and requirements:

• Opt-in to building the standard library instead of using pre-built artifacts.
• Specifying alternate standard library implementations.
• Create a way for Cargo to pass --extern=foo to rustc to specify explicit dependencies to add to the extern prelude, even for pre-built artifacts. This can help obviate the need for extern crate for crates like proc-macro.
  • We may also consider this to be implicit in some cases. For example, proc-macro could be automatically added with proc-macro = true.

There have been a few different proposals for syntax, please see the following:

• RFC 1133 proposal
• RFC 2663 proposal
• #5002 — Explicit standard library dependencies
• #5003 — Implicit standard library dependencies
• #57288 — Tracking issue for no more extern crate for sysroot crates

Some issues to consider:

• How does cargo treat multiple crates in a graph declaring dependencies on the standard library?
• How to balance implicit vs explicit dependencies?
• How to express a dependency on a pre-built artifact vs building one from source.

[SimonSapin]

Some thoughts:

• How does cargo treat multiple crates in a graph declaring dependencies on the standard library?

Same as when multiple crates in the graph declare a dependency on a given crates.io crate: unify them. Except that the standard library doesn’t have a version number that is meaningful to Cargo, so the version key should not be allowed and you can’t end up with two copies because versions incompatible with each other were requested.

For feature flags specifically (https://github.com/rust-lang/wg-cargo-std-aware/issues/4), this means taking the union of all requested features.

• How to balance implicit vs explicit dependencies?

This one is tough because we’re stuck with what’s already stable. Most existing stable crates depend on libstd implicitly, so it’s tempting to declare that every crate does unless it opts out.

But we also have existing stable #![no_std] crate that do not have this explicit opt out (since the opt out mechanism doesn’t exist yet), but are still compatible with targets that do not have a libstd at all. Cargo needs to continue to support those as well.

• How to express a dependency on a pre-built artifact vs building one from source.

I think there should not be direct control over this. Cargo should decide based on other factors what libstd it wants (profile settings https://github.com/rust-lang/wg-cargo-std-aware/issues/2, target https://github.com/rust-lang/wg-cargo-std-aware/issues/3, feature flags https://github.com/rust-lang/wg-cargo-std-aware/issues/4, etc), then see if a binary is already available with that configuration, and if not fall back to compiling it.

Any difference between a binary shipped through rustup and one rebuilt with the same configuration is a bug. Perhaps a reproducibility bug https://github.com/rust-lang/rust/issues/34902, or perhaps another criteria should be added to this "configuration" concept.

[Ericson2314]

This one is tough because we’re stuck with what’s already stable. Most existing stable crates depend on libstd implicitly, so it’s tempting to declare that every crate does unless it opts out.

Just doing that is the easiest plan.

But we also have existing stable #![no_std] crate that do not have this explicit opt out (since the opt out mechanism doesn’t exist yet), but are still compatible with targets that do not have a libstd at all. Cargo needs to continue to support those as well.

Hmm on platforms with std the extra implicit dependency is fine. On platforms without std...I hope we can quickly transition most crates, and seeing that those platforms have the most to gain from this the breakage will be worth it.

The only other alternative I see would be making a breaking change in the Cargo.toml format (requiring all deps explicitly) and allowing Cargo to parse old versions. That's nice, though crates using the new format drop do support for old rustc/cargo.

Trying to have Cargo "fish out" the #![no std] sounds like a nightmare. I absolutely recommend against that. [If anything, I rather go in the opposite direction where #![no std] becomes as deprecated as extern crate.]

I think there should not be direct control over this.

Amen!

[jethrogb]

Opt-in to building the standard library instead of using pre-built artifacts.

This seems like something that shouldn't just be part of the regular dependency specification.

[PoignardAzur]

But we also have existing stable #![no_std] crate that do not have this explicit opt out (since the opt out mechanism doesn’t exist yet), but are still compatible with targets that do not have a libstd at all. Cargo needs to continue to support those as well.

A thought occurs: maybe it would be a good thing to add a no_std flag to cargo right now, even if that flag doesn't do anything yet, so that when a later version comes out and does use the flag, crates can use it without bumping MSRV.

(my understanding is that was the reasoning for the rust-version flag?)

[tarcieri]

An idea I posted to IRLO awhile ago: https://internals.rust-lang.org/t/pre-pre-rfc-making-std-dependent-cargo-features-a-first-class-concept/10828

Pave the cowpaths we have today, making the alloc and std features first-class (probably at an edition boundary).

They could be unconditionally linked by default just like today, but adding a std feature at all could automatically make a crate no_std, and when the std feature is enabled, automatically link std the same way specifying extern crate std would in a no_std crate, and ditto for alloc.

This requires no new syntax or concepts, just semantic changes that would eliminate the existing boilerplate.

Existing code would work just fine, but linters could warn you existing no_std and extern crate std directives are no longer necessary and redundant.

Bonus points: this gets rid of the last remaining use cases for extern crate, I believe.

[Lokathor]

first of all extern crate is still useful for forcing a crate to be linked in.

but second, some crates are no_std but don't have a std feature (or alloc feature), so you must design around that case too.

[tarcieri]

Yeah sure, but the boilerplate for #![no_std] along with a feature-gated extern crate std in the event you want to support both a no_std and std profile is a lot which it’d be nice to replace with declarative first-class features, the same way the 2018 edition removed the need for routine extern crate for every crate dependency.

Also sidebar, but the no_std approach is very much antithetical to the additive nature of features: it’s something you add which subtracts functionality. This has lead to countless bugs I’ve had to endure where people either add a subtractive no_std crate feature or add broken cfg(not(feature = “std”)) gating leading to code which is broken when the std feature is disabled.

So even if you don’t like my suggestion for first-class alloc and std crate features, I would implore you to make the Cargo linkage to libraries like alloc and std linking additive rather than subtractive, just like crate features, with defaults that can be removed/overridden, rather than having to actively declare what you don’t want.

[Rahix]

first of all extern crate is still useful for forcing a crate to be linked in.

@Lokathor

use crate_name as _;

or just

use crate_name;

also works fine for that purpose.