(Modules) Tracking issue for Picking a Module Path System variant

[Centril]

This is a sub-tracking issue for the RFC "Clarify and streamline paths and visibility" (rust-lang/rfcs#2126) as well as the internals post Relative Paths in Rust 2018.

The issue deals with revamping paths and making a decision about whether we should pick:

• The uniform_paths variant. Proposed in Relative Paths in Rust 2018.

• The anchored_use_paths variant. You have to use self::, super::, and crate::, or an external crate name explicitly.... See rust-lang/rfcs#2126).

• [x] Implement all the things.

  • [x] uniform_paths, https://github.com/rust-lang/rust/pull/52923
  • [x] anchored_use_paths

• [x] Make both variants available to test in Edition Preview 2 with appropriate documentation on how to test.

• [x] Adjust documentation (see instructions on forge)

• [ ] Pick a variant

• [ ] Stabilization PR (see instructions on forge)

Unresolved questions:

• [ ] Which variant do we pick?

[sanmai-NL]

Could you give more details about what has been implemented concretely, and what renaming operations are needed exactly? It comes down to: how can I start testing anything right now?

A priori, I vote for anchored_use_paths, for being the most explicit/teachable. One suggestion: I find the crate keyword a bit odd since it does not seem related to visibility at all. I suppose this has been debated before but it would be better if it becomes pubcrate or whatever.

[Centril]

cc @eddyb re. @sanmai-NL's question.

[eddyb]

@sanmai-NL Now that #52923 has landed, the next nightly should have both options available:

• anchored_use_paths has been the Rust 2018 default for some(?) time now
• uniform_paths you get by being on Rust 2018 and adding #![feature(uniform_paths)]

I'd suggesting trying the latter after upgrading to the edition, and report back if the ambiguity errors end up being too excessive.

AFAIK, the upgrade involves taking a Rust 2015 crate, adding #![feature(rust_2018_preview)] without changing the edition, running rustfix to apply the suggestions it gives for making same-crate paths explicit (i.e. prepending all paths that don't start with a crate name, with crate::), and only then do you switch the edition (usually in Cargo.toml) to Rust 2018. You can remove #![feature(rust_2018_preview)] afterwards, since it's redundant.

[joshtriplett]

https://github.com/rust-lang-nursery/edition-guide/pull/85

[archer884]

I absolutely despise the "anchored paths variant," and I am very excited to see that we're considering an alternative. Any alternative, honestly. >.>

...That said, uniform paths look great. :) I'd much prefer to let people use absolute paths (the anchored version) if they like, but not force people to do it.

(No idea where the appropriate place to provide feedback for this would be, so I just left it here. Sorry.)

[kohensu]

Using uniform_paths, there is an error I find confusing. Considering:

#![feature(rust_2018_preview)]
#![feature(uniform_paths)]
use hex;
fn main() {
    println!("hex {}", hex::encode("01"));
}

I get this error:

error: import from `hex` is ambiguous
 --> src/main.rs:3:5
  |
3 | use hex;
  |     ^^^
  |     |
  |     could refer to external crate `::hex`
  |     could also refer to `self::hex`
  |
  = help: write `::hex` or `self::hex` explicitly instead
  = note: relative `use` paths enabled by `#![feature(uniform_paths)]`

whereas self::hex doesn't exist in the module.

Because the code is working without use hex maybe suggesting to remove it will be more clear.

[Nemo157]

@kohensu see https://github.com/rust-lang/rust/issues/53408, I think the plan is to either special case allow that or improve the error message to just suggest removing it.

[kohensu]

@Nemo157 Sorry, I did not see that. Thanks for the reference to the issue.

[sanmai-NL]

@archer884: What’s more interesting is your motivation to feel that way?

[Xymist]

As a user (who perhaps hasn't been paying as much attention as I should) I think the uniform_paths option feels strictly better. Special case sugar for when there's no possible ambiguity would be nice, but by no means necessary; optimising for clarity is definitely a greater win.

[eddyb]

Special case sugar for when there's no possible ambiguity would be nice

Not sure what you mean. Does #53427 (by treating each namespace independently) solve that for you?

[archer884]

@sanmai-nl:

Either Eric Lippert or Anders Hejlsberg had an article on this awhile back: people want loud syntax for things at first, but then as time goes on they want the syntax to be unobtrusive and elegant instead. If we make long, annoying imports the standard (and ONLY) option, we will absolutely regret it.

"Maybe not today, maybe not tomorrow, but someday." :)

[sanmai-NL]

That’s an argument from authority, and a very aspecific one. Conciseness is a criterion independent from explicitness and also from simplicity.

[archer884]

@sanmai-NL, it's not. It's just a chance for you to read the thoughts of another person who covers the issue in greater depth. Google is your friend.

[sanmai-NL]

Very poor way to argue your case. You seem not to even recall which author you mean, but you ask me to Google for something.

[archer884]

Ok, I'm done being badgered here. Glad to see rust is still so inclusive. :)

On Fri, Aug 17, 2018, 9:42 AM Sander Maijers notifications@github.com wrote:

Very poor way to argue your case. You seem not to even recall which author you mean, but you ask me to Google for something.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rust-lang/rust/issues/53130#issuecomment-413887752, or mute the thread https://github.com/notifications/unsubscribe-auth/AApeRmCERlVvJYp9wVS9mBYTeJaUolXMks5uRtZFgaJpZM4VxEE_ .

[liigo]

cc me

[sanmai-NL]

Sad to see how some people dramatize a technical discussion. I hope only comments that are motivated and consider the earlier discussion are taken seriously. Just chiming in doesn’t bring us any further.

[eddyb]

@liigo You can also use the "Subscribe" button on the right, without posting a comment.

[luben]

Is there a reason why use core::mem does not work?

  |
1 | use core::mem;
  |     ^^^^ Did you mean `self`?

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: Could not compile `uds`.

Aside, I see that #53427 fixed use log::* but I hit the same ambiguity with use simple_logger:

   |
14 | use simple_logger;
   |     ^^^^^^^^^^^^^
   |     |
   |     refers to external crate `::simple_logger`
   |     defines `self::simple_logger`, shadowing itself
   |
   = help: remove or write `::simple_logger` explicitly instead
   = note: relative `use` paths enabled by `#![feature(uniform_paths)]`

Aren't the modules a separate namespace?

[joshtriplett]

@luben For your second comment: just delete the line use simple_logger; entirely. extern_prelude makes that redundant; you can just call simple_logger::init() or similar.

@eddyb Should the special-case version of that message reference extern_prelude instead? Because it's extern_prelude that makes use cratename; redundant.

[eddyb]

@luben You need to use #![no_std] to get core instead of std. Any particular usecase for accessing core without #![no_std]?

[luben]

Thanks for both the responses. Makes sense. Actually I didn't need core::mem as I can use std::mem instead: I have copy/pasted some futures related code that turns to work without uniform_paths.

[Nemo157]

Any particular usecase for accessing core without #![no_std]?

@eddyb futures-rs uses the opposite (accessing std with #![no_std]) a lot for supporting an optional std feature, a good example is futures-core::stream, this has all the core supported code first, then near the bottom the allocation using code behind a #[cfg(feature = "std")] gate.

This works in 2015 by having an unconditional #![no_std] along with #[cfg(feature = "std")] extern crate std; to optionally add the crate. It also worked in 2018 anchored_use_paths with an unconditional #[no_std] and no explicit import because libstd was available in the search path (see https://github.com/rust-lang/rust/issues/53166#issuecomment-411196219).

I'm not sure how to support this as easily with uniform_paths. Adding #[cfg(feature = "std")] extern crate std; to the root requires updating all the use std::*; imports to use crate::std::*;. Using #![cfg_attr(not(feature = "std"), no_std)] requires switching all imports between use core::*; and use std::*; based on the feature. Out of these two options I think the first is definitely the most reasonable, but that requires extern crate which may be deprecated in 2018, and it's not quite as nice as the old system of having normal looking core/std imports everywhere.

EDIT: While editing this post I managed to drop one of the main points. The solution I'd most be interested in seeing is exactly what @eddyb asked about above, being able to access core even without #![no_std]. That way futures-rs could use #![cfg_attr(not(feature = "std"), no_std)], when the std feature is inactive it would only import via core rooted paths, when std is active it would import via both std and core rooted paths.

[eddyb]

You can just write use ::std::... and it still works with uniform_paths, btw. That syntax works with ::core::... as well, too, irrespective of #![no_std]. (paths starting with :: explicitly look up a crate and are not limited to the extern_prelude filter)

[joshtriplett]

@eddyb Is there any particular reason not to have core accessible even without #[no_std]?

Doing so allows more easily writing code that works with or without #[no_std], if you only use functions in core.

[eddyb]

@JoshTriplett Not really any that I can think of (maybe modules named core causing ambiguities?). We might want to keep only one to avoid confusion over which to use, but it also seems useful to have both. I wonder what @rust-lang/lang and @rust-lang/libs think of allowing core, as well as std, to be used everywhere without #![no_std].

[Centril]

I think allowing core everywhere sounds like a good idea. In particular, it seems to me like it would make it easier to transition to #![no_std] support.

[Nemo157]

How would it affect the plans for a stable alloc crate, that RFC is based on using extern crate?

I assume a std-aware cargo would subsume this functionality, and you could explicitly list which facade crates you depend on in your Cargo.toml (hopefully with optionality based on features).

[petrochenkov]

The question of core in prelude was briefly discussed recently in https://internals.rust-lang.org/t/2018-edition-end-of-week-post-2018-07-20/8019. I see no problems with adding it.

The only restriction that, I think, we need to enforce is not adding crates that are not dependencies of std into prelude (e.g. proc_macro), and not adding crates that are not dependencies of core when no_std is enabled. This way we can avoid special-casing these crates in the compiler and just put them into core/std library preludes. This actually may be possible right now since https://github.com/rust-lang/rust/pull/52923 fixed some bugs in single-segment imports.

[petrochenkov]

I haven't seen one more option mentioned.

--extern currently has the next syntax:

--extern NAME=PATH

We can make the PATH optional

--extern NAME

and search the crate NAME in library search directories in this case, and if it's not found then it's an error. The crate NAME would still be added into extern prelude.

Example:

// `proc_macro` is available in extern prelude and is searched in library search directories
rustc --extern=proc_macro lib.rs

[eddyb]

If we have a mechanism like --extern proc_macro we could add Cargo support like this:

[dependencies]
proc_macro = "builtin"
# expanded form (should we allow version constraint here?)
proc_macro = { builtin = true }

Although it seems more bikesheddy than the --extern side. EDIT: nevermind, I forgot the name = "1.2.3" shorthand also works like this

[joshtriplett]

@eddyb Big fan of that syntax, and the --extern NAME syntax.

[alexreg]

Same.

[Nemo157]

With a potential extension to replace no_std:

[dependencies]
core = "builtin"
std = { builtin = true, optional = true }

Although this will conflict with having a std feature that activates other dependencies features... (but hopefully that can be fixed for optional dependencies in general) and my first thoughts on implementation seem difficult to make backwards compatible with existing no_std crates.

[eddyb]

@Nemo157 Ohhhh right, having both name = "foo" and name = { foo = true } is fine! I updated my original comment, I like this a lot.

cc @rust-lang/cargo

[mthgr]

Absolute beginner perspective: I find the anchored_use_paths clearer and easier to understand. I've been dabbling in and out of Rust for over 2 years now reading all the articles I can find etc. Admittedly, I did very little actual coding other than to try the exercises in books etc. So take my opinion for what it's worth... I think it's more approachable especially in conjunction with all the other new changes. Keep up the good work.

[withoutboats]

We discussed this in the cargo meeting today and felt that we should not make a decision for the unstable crates now, and that we should put proc_macro in the prelude. Possibly, we should rename it.

The other crates are all unstable, and we can leave them requiring extern crate for now.

[SimonSapin]

we should put proc_macro in the prelude

What does this mean? Most crates do not want to link to the compiler.

Possibly, we should rename it.

proc_macro and core are stable.

[Havvy]

@SimonSapin You can rename the re-export in the prelude.

[eddyb]

@withoutboats Uff, but for non-proc_macro crates, I feel like we should limit it, and not load arbitrary things off the disk just because they happen to "fit". Maybe someone has an alloc or syntax or test crate-level module and they accidentally put :: in front out of it (e.g. out of habit), now they're getting some random internal crate!

Maybe I should've poked around and asked to join the Cargo team meeting, although I just got more ideas that would work well.

Specifically, we can teach rustfix to add e.g. rustc_driver = "builtin" to Cargo.toml, right? Oh and it should be "bundled" instead, IMO. A lot of the crates that would be loaded this way don't really feel "builtin".

Also, can we make it an unstable Cargo feature and not worry too much about it? Since these are all unstable crates. We can stabilize --extern name instead for people not using Cargo, that seems as least hazardous as it gets.

@SimonSapin

What does this mean? Most crates do not want to link to the compiler.

Assuming we can land #49219 in the next week or so, we wouldn't care.

proc_macro would be just an rlib with only a dependency on std, nothing more. In that PR, libtest depends on it (as a hack), so if we build libtest for a target, we get proc_macro too.

We should probably just put proc_macro in the extern_prelude of --crate-type=proc-macro, nothing should realistically break then (oh but libraries like proc-macro2 depend on it... can we get Cargo to pass --extern proc_macro for all of those dependencies? or is that a bad idea?)

Maybe we should just try adding proc_macro to the prelude for everyone on Rust2018 without no_std and see what happens. It shouldn't cause any problems (worst case, uniform_paths ambiguities), at least after #49219.

[LukasKalbertodt]

I agree with @mthgr: I really like anchored_use_paths. They make reading code much clearer. I often tried to read through foreign code bases and the imports were all over the place. So I had to manually check which extern crates are added and which modules exist.

Also, I don't get the argument about being "verbose" or hard to write. I have been using nested imports since they landed (and I don't understand why nested imports aren't used everywhere by now^{c'mon people, less repetitionlet's not discuss this here}). And with nested imports, using use crate::{...} can actually be shorter than or at least as short as the old non-anchored_use_paths variant. Example with three modules:

// old
use cat::{Ear, Nose, meow};
use cheese::{Color, Holes};
use dog::GodBoy;

// new
use crate::{
    cat::{Ear, Nose, meow},
    cheese::{Color, Holes},
    dog::GodBoy,
};

Sure, it's two lines more, but I also didn't need to repeat the boring use keyword. And I obviously also didn't need to repeat crate:: for each import.

[eddyb]

@alexcrichton Would it be realistic to turn off "plain" -L in the new edition (IIUC, -L deps= is needed by Cargo and we don't have a way around it)? We can tell people to migrate to --extern foo=... (we should even be able to print out where Rust2015 would find each of them, just not allow it by default anymore) and keep --extern foo unstable, for sysroot crates?

Eventually, if we want to stabilize, say, alloc, we could stabilize a proper Cargo.toml mechanism.

[joshtriplett]

cc @rust-lang/cargo

Regarding crates in the sysroot, in the Cargo meeting we agreed that long-term we want to find a solution for those that involves explicitly specifying them in Cargo.toml. However, there was some hesitation to adopt something like proc_macro = "builtin" or proc_macro = { builtin = true }, primarily because people wanted something that would align with the future xargo-style integration of std and core, with a syntax that would allow non-builtin versions of all these crates too. So, in the interim, we felt like it made sense to expose only the stable crates in the sysroot (std, core, and proc_macro), and avoid exposing the unstable crates (like test), which means that stable code can avoid using extern crate, while unstable code still has to use extern crate for a little longer until we come up with a good general syntax for handling such dependencies.

[petrochenkov]

As I previously said in https://github.com/rust-lang/rust/issues/53130#issuecomment-414615070, I'm wary of adding proc_macro into extern prelude because proc_macro is not a dependency of std and this means it cannot be added to std prelude, so this means it has to be hardcoded into the compiler.

I made an experiment yesterday and confirmed that use crate as std; placed into rust/src/libstd/prelude/v1.rs works as expected without any compiler hardcoding. (Except that imports with feature(uniform_paths) can't look into the library prelude yet, but that needs to be fixed if the paths are supposed to be actually uniform.)

[eddyb]

@petrochenkov How is that supposed to work, would extern_prelude remain solely for explicitly supplied --extern crates, and core / std would move to the 2018 prelude?

Where would use std; resolve? Unshadowed self::std from the use prelude::* import? Would explicit self::std actually work, as well? Right now it doesn't/shouldn't. What would the relationship between --extern crates and contents of the prelude be?

---

I just looked at #49219 again and the main things it needs from std are:

• String, Vec, BTreeMap (available in alloc)
• HashMap, thread_local!, sync::Once and panic control (only in std)

Some of this code is server-side-only so maybe it can be moved to a separate unstable crate.

However, the client::BRIDGE thread_local! and the Once for the panic control must be in the same crate - we can maybe use #[thread_local] static for that, and/or parking_lot? The big problem is the panic control itself, because proc_macro must catch and silence panics entirely client-side (the server could be using a separate libstd, and only talks to the client through the bridge).

So I don't think it's feasible to make proc_macro a dependency of libstd, in the short term. (cc @alexcrichton)

[eddyb]

@joshtriplett Is that implying ::foo should not load non-extern_prelude crates, and extern crate must be kept around instead?

What about using an unstable Cargo.toml feature to handle e.g. rustc_driver = "bundled"?

[joshtriplett]

@eddyb I was under the impression that it already worked that way in the current version. Does it not?

The proposal was that extern_prelude would include all the crates passed via --extern and all the stable sysroot crates, and those would be the only ones that worked via use and ::foo; unstable sysroot crates would still need extern crate for now, and we'd make a plan to handle them post-edition.

I'd like to have a Cargo.toml syntax for that (corresponding to the rustc syntax --extern foo without an =path), but that would need forward compatibility with std-aware cargo.

[eddyb]

I was under the impression that it already worked that way in the current version. Does it not?

Nothing changed, just use foo::...; (without leading ::) under uniform_paths, considers only extern_prelude, leading :: paths with and without uniform_paths are like extern crate still.

but that would need forward compatibility with std-aware cargo

That's only if it's stable, right? But Cargo now has unstable features (e.g. edition and default-run).

[petrochenkov]

@eddyb

How is that supposed to work, would extern_prelude remain solely for explicitly supplied --extern crates, and core / std would move to the 2018 prelude?

Yes. (std and core can be added to the 2015 library prelude too though.)

Where would use std; resolve? Unshadowed self::std from the use prelude::* import?

With uniform_path? Into name std from the prelude (not from any module). #[prelude_import] use ... doesn't plant anything into a module. It's not even a use item really, just some unstable vaguely appropriate syntax to put the attribute #[prelude_import] on.

Would explicit self::std actually work, as well? Right now it doesn't/shouldn't.

No, it wouldn't since #[prelude_import] doesn't plant anything into a module.

What would the relationship between --extern crates and contents of the prelude be?

Unrelated? Two preludes interact slightly in the sense that names from extern prelude shadow names from std library prelude as described in this comment.