search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
96.97k stars 12.53k forks source link

(Modules) Tracking issue for `crate` as a visibility modifier #53120

Closed Centril closed 2 years ago

Centril commented 6 years ago

This is a sub-tracking issue for the RFC "Clarify and streamline paths and visibility" (rust-lang/rfcs#2126) dealing with the question of crate as a visibility modifier.

Unresolved questions:

Centril commented 6 years ago

Comments about crate as a visibility modifier:

Parsing ambiguity

Unnatural / Confusion / Not improvement

A good idea

pub(extern) instead

Bikeshed

An early preview

Dedicated thread

Centril commented 6 years ago

Personally, I am very much in favor of crate as a visibility modifier and I share @stepancheg's comment here. I think that we should encourage smaller and stricter visibilities and crate does just that.

lilianmoraru commented 6 years ago

I'm personally ok with writing pub(crate), the intent seems very explicit. The example given by @johnthagen is really painful to see(using crate):

use crate::menu::{Sound, Volume};

crate mod color;

...

/// A type for storing text and an associated color it should
/// be drawn as.
crate struct ColoredText {
    crate color: types::Color,
    crate text: &'static str,
}

crate mod color; especially seems confusing, you definitely need to put a bit of thought on what's happening there.

rpjohnst commented 6 years ago

Some of these examples are very C-static-esque- intuitively related but really surprisingly distinct.

Centril commented 6 years ago

The example due to @johnthagen does not read poorly to me. In fact, it reads naturally and I quite like the symmetry. It is beautiful in a way.

If the readability of:

crate struct ColoredText {
    crate color: types::Color,
    crate text: &'static str,
}

becomes a problem; then an IDE/editor that understands the syntax of Rust can highlight the crate tokens in the different positions with different colors. That should clear up the difference well I think.

matklad commented 6 years ago

crate as visibility modifier is definitely weird: it uses very rust-specific keyword for a thing which is not specific to rust. Kotlin & C# use internal for this.

I'd personally would want to reuse pub for crate-visible, and go for a more screamy syntax for world visible, like pub* or pub!.

johnthagen commented 6 years ago

It's been brought up before, but I think the root issues I see are:

  1. crate is a noun. I think pub(crate) is long and odd looking, so I fully support replacing it with something, but it did have the public adjective associated with it, so grammatically it flowed better.
  2. crate is now used as the anchor for "this-crate" imports, which means something different than "wherever this is defined, it's also exported visibly from this crate"
// Here `crate` means the root of this crate.
use crate::menu::{Sound, Volume};

// Here, `crate` means: export crate::game::color
// The `crate` is referring to `color`, not the root.
crate mod color;

...

/// A type for storing text and an associated color it should
/// be drawn as.
// Same potential confusion as `crate mod color`
crate struct ColoredText {
    crate color: types::Color,
    crate text: &'static str,
}

Compared to an example, using @matklad's internal from Kotlin/C#.

use crate::menu::{Sound, Volume};

internal mod color;

...

/// A type for storing text and an associated color it should
/// be drawn as.
internal struct ColoredText {
    internal color: types::Color,
    internal text: &'static str,
}

I'm not saying internal is the right keyword (Rust likes it's very short abbreviations, and int is unfortunately full of C/C++/Java confusion), but I personally think that the second example is immediately more readable.

I also think the crate visibility keyword will be confusing to people coming to Rust from other languages. If even many of us involved enough in Rust to comment on these threads are jarred, I'd have to imagine it will trip up people new to Rust as well.

seanmonstar commented 6 years ago

The slightly longer syntax of pub(crate) may not be such a big deal if weren't becoming a warning/error to have pub items that aren't reachable outside the crate. I personally wish that if I had a pub(crate) struct Foo { ... }, that the compiler could realize that all the pub fns in a impl Foo are clearly not reachable, and not bother me about it.

I find it to be just busy work currently in Rust 2015 if I ever mark a type from pub struct Foo to pub(crate) struct Foo, how the compiler then yells in all the places that some other pub fns exist using the suddenly pub(crate) type, when the problem isn't real, because the other type is also pub(crate).

johnthagen commented 6 years ago

I also find @matklad's idea of re-purposing pub as crate-public and using export or something for world-visible exports. But that may be too big a divergence for an edition?

rpjohnst commented 6 years ago

Repurposing pub as crate-public and adding a new visibility for world-public was the proposal before the current version. Such a change to existing semantics was considered too drastic even for an edition, which is why pub is now keeping its current meaning.

What was less discussed and considered, I think, was repurposing pub solely via a lint. Perhaps we could switch the lint from "warn on pub that's not accessible outside the crate" to "warn on pub that is accessible outside the crate," and add a purely optional pub(extern)/export keyword. That is, don't change any semantics, just add a lint-silencing syntax.

I suspect this would be less disruptive, based on the assumption that there are fewer world-public items than crate-public items. It would also preserve the intuitive (though subtly incorrect) meaning of pub as "export from the current module" rather than confronting everyone with visibility's true behavior.

glaebhoerl commented 6 years ago

Rust likes it's very short abbreviations, and int is unfortunately full of C/C++/Java confusion

FWIW, though it only saves two characters, if we wanted to abbreviate internal, the "right" abbreviation would probably, by analogy with external, be intern. Unfortunate that that is also a noun with a commonly understood, different meaning. Oh well.

johnthagen commented 6 years ago

@glaebhoerl intern is a good option to consider! ❤️

The symmetry with extern is really nice, and IMO would greatly reduce the potential confusion with the noun form of intern.

It's short, (only 1 more character than crate) and doesn't clash with use crate::.

The updated example would look like:

use crate::menu::{Sound, Volume};

intern mod color;

...

/// A type for storing text and an associated color it should
/// be drawn as.
intern struct ColoredText {
    intern color: types::Color,
    intern text: &'static str,
}
CasualX commented 6 years ago

I've mentioned it before, but I'm not sure what the problem is with nouns used as adjectives?

To recap: there are plenty of nouns that can be used as adjectives, eg. a house cat, a computer mouse, a computer desk, etc... A google search for english nouns used as adjectives seems to indicate there's nothing inherently wrong although not all nouns work as adjectives.

Let's try it:

crate mod hello; A crate module named hello, feels fine. crate fn world() {} A crate function named world, feels fine. crate struct Foo; A crate struct named Foo, feels fine. crate enum Bar {} A crate enum named Bar, feels fine. crate trait Baz {} A crate trait named Baz, feels fine.

crate use self::local::Foo; Ack, this one does not work, a crate use? You can read it as crate usable item named Foo. It does break the pattern.

It can also be awkward when used in front of struct members and even more in combination with crate as the root of a path.

While crate isn't perfect, I'm not convinced that 'being a noun' is a deciding factor.

UtherII commented 6 years ago

The problem is that it is very uncommon. There is no programming language I know that use nouns as type modifier.

epage commented 6 years ago

@Centril

becomes a problem; then an IDE/editor that understands the syntax of Rust can highlight the crate tokens in the different positions with different colors. That should clear up the difference well I think.

Personally, while I find features of different editors nice, I don't think we should design the language with the assumption of a sufficiently advanced editor. I felt like C# was designed this way and it was a major factor in my frustration with that language.

Centril commented 6 years ago

@epage I think crate as a visibility modifier is a good idea regardless of highlighting; I'm merely suggesting that highlighting is additional mitigation. In particular, it should be fairly trivial for any editor to highlight crate:: differently than crate field because the former is always crate + :: which is easy to check for in all cases except crate ::foo::bar (but this will be fairly rare..).

matklad commented 6 years ago

As an IDE person, I think such highlighting would add significant amount of noise for a very little amount of information, netting negative. IMO (this is highly personal, but informed by both using and implementing powerful IDEs) highlighting works best when it conveys semantic non-local information (does this usage refers to a variable that was declared with mut) and de emphasizes local "boilerplaty" aspects of code (so all keywords should be styled exactly the same).

zesterer commented 6 years ago

It seems to me that dom (i.e: domestic) is a potential candidate.

It has three letters (nice for alignment, easy to remember), it's not a noun, and - other than 'Document Object Model' - I don't think it has any particular ambiguity attached to it.

pub struct MyStruct {
    dom num: i32,
    pub msg: String,
}

Does anybody have thoughts on this?

epage commented 6 years ago

One angle on this that I've seen mentioned but couldn't find in the summary (thanks for doing that btw!) is how a shortcut fits in with the existing pub() syntax.

If pub and <something> (e.g. crate) have special meaning, it further reduces the visibility, and by extension familiarity, of pub(<something>). Whatever solution we go with, I think it should support or replace the existing machinery rather than being yet another one.

For example, if we use crate or replacement:

Considering this and my understanding of the goal (clarify public API from internal API), led me to recreate @vitiral's idea of pub(extern).

epage commented 6 years ago

RE Impact on encapsulation

One benefit of the existing pub system is encapsulation. The easy-path is to expose API only one-level up. This makes it easier to have things public to parts of a crate but private to the whole create.

While there will still be pub(super), having a shortcut for pub(crate) will push people in the direction of using it more, encouraging people to not encapsulate their APIs.

I suspect this isn't an issue because of the culture of small crates.

But in considering this, it gives me another iteration on my above comment on pub(extern)

I previously brought up the concern of people transitioning from other languages. This better aligns with them.

imo this is the best of all worlds. So tear it apart and help me understand why not :)

parasyte commented 6 years ago

I still hate the pub(foo) syntax. Hyperbolically, it looks like it can't decide if it's a function call or a mashup of multiple keywords. We don't use let(mut) or for(in) so what's the deal with this one?

vitiral commented 6 years ago

@parasyte pub<foo> for the win! After all, isn't it a type of visibility?

lu-zero commented 6 years ago

pub<crate> or pub(crate) feel better indeed.

ralfbiedert commented 6 years ago

Some thoughts from someone who changed camp:

At first I was very opposed to crate and thought "this is ruining the nice pub".

I then tried it side-by-side in some of my projects and let it sink in.

Frankly, after a few days I couldn't stand looking at pub(X) anymore, it feels clunky in comparison, and harder to read.

Initially I feared there might be (visual) ambiguity; but to me the opposite happened: If I see crate now I know it's, well, "crate stuff". Whether that is importing modules or declaring visibility. What exactly is in the overwhelming majority of cases very clear from the context (sort of like ambiguity in English).

I can see there might be still be residual "harder" (visual) ambiguity in some cases, but I wouldn't want to trade that back for what now feels like an massive quantitative readability win (as in: "source code lines that require less visual tokenization / effort vs. source code lines that became more ambiguous" ).

From that angle crate - intern (or any other asymmetry) would also feel like a step back.

Having said this, I don't know about parsing ambiguity. If I had to pick one, I'd much rather have a good story around "crate means crate stuff" than a good story around "crate ::foo::bar just works".

gnzlbg commented 6 years ago

My two cents are that:

If it were up to me, I would use vis as the keyword, and the type of visibility as the modifier, e.g. vis(pub), vis(crate), etc. because this makes more sense to me.

Given that we are already stuck with pub as a "visibility specifier", I actually like crate. The pub(crate) reads for me as public to this module, private to the crate - I find using both public and private at the same time here weird.

Introducing new keywords and contextual keywords, etc. is in my opinion not worth it. Teaching that there are two visibility modifies pub and crate, and that one means public, and the other means private to the crate, kind of makes sense to me, but maybe I just got used to crate already during the last two weeks.

zesterer commented 6 years ago

To those that suggest an already used keyword (i.e: crate) conflates meanings, I'd argue that context is more important than the word itself. The brain parses everything with context (how the compiler parses this is a different matter): this explains why we don't conflate the semantic meaning of for in for x in y and impl X for Y.

Equally, introducing crate as a visibility qualifier would likely not create confusion because its meaning, in the context of a member or function qualifier, is obvious when provided with that additional context. For example, crate fn my_func(); doesn't read as "this is a crate", it reads as "this is a crate-visible function".

That said, the fact that it is a noun is inconsistent. I'd be for creating a new visibility qualifier keyword to address this issue.

In fact, if there's anything that definitely will confuse users, it's the pub(crate) syntax which intuitively looks like a function call and has no other syntactic equivalent elsewhere in the language. To me, it feels like an ugly hack.

johnthagen commented 6 years ago

As someone who's brought up concerns about using crate as the replacement for pub(crate) and after reading @aturon 's latest post:

Supporting crate as a visibility modifier (tracked here 138. Given feedback so far, this feature is unlikely to be stabilized for Rust 2018.

I just want to make sure I'm clear that I am personally fully in favor of replacing pub(crate) with something (as I think the majority is).

In order of preference, with what I feel would be most easy to grasp, especially for those new or unfamiliar with Rust:

  1. intern (or some other similar new keyword)
  2. crate
  3. pub(crate)

If the core team feels like intern or something similar would ultimately never be accepted, then I'd get behind crate as I still think it's a big improvement over pub(crate), for the reasons @Centril and others have articulated.

So I wouldn't stand in the way of this if the, much more experienced, core team feels this is the best path forward. It's neat to just be able to provide feedback express ideas for consideration. 👍 Rust!

epage commented 6 years ago

@ralfbiedert

Initially I feared there might be (visual) ambiguity; but to me the opposite happened: If I see crate now I know it's, well, "crate stuff". Whether that is importing modules or declaring visibility. What exactly is in the overwhelming majority of cases very clear from the context (sort of like ambiguity in English).

@zesterer

To those that suggest an already used keyword (i.e: crate) conflates meanings, I'd argue that context is more important than the word itself. The brain parses everything with context (how the compiler parses this is a different matter): this explains why we don't conflate the semantic meaning of for in for x in y and impl X for Y.

I'm not meaning to call you two out personally but serve as examples of the people in favor of the change after using it.

While I find it weird looking, my biggest concern is with non-rustaceans.

I'd love it if we could do usability studies on this to know better how much impact these concerns have.

ralfbiedert commented 6 years ago

@epage, very much agree, the user-facing parts of Rust should be UX tested. However, I think this is what is happening right now, and we are in the middle of discussing the results.

To add to that, some anecdotal observations from our company:

I am the "Rust advocate" in our department, working with 3 others. All have a solid background in C#, but are relatively new to Rust. The other day I migrated our research project to Rust 2018, along with the "crate-stuff".

When we went through the code, the conversations went approximately like this:

"So here are some other changes I made; new import system; modifiers."

"What does that do?" (pointing at use crate::object and crate x: object)

"Import from this crate." and "Visibility modifier."

"Ah, ok. Anything else that changed?"

End of discussion.

Surely, we chatted a bit more about new features, and whether we should adopt certain patterns; but the "teaching aspect" of these two items boiled down to a few words and has not been mentioned since (to my knowledge).

If I hear anything else the next time we talk I will update this comment.

epage commented 6 years ago

@ralfbiedert Thanks for sharing that!

@epage, very much agree, the user-facing parts of Rust should be UX tested. However, I think this is what is happening right now, and we are in the middle of discussing the results.

While I value these UX anecdotes, especially for the learnability side, I don't think I'd qualify this issue's process as UX testing. My comment is referring to a more formal process that can help gain deeper understanding, weed out biases, etc.

superseed commented 6 years ago

Here are a few thoughts coming from a Rust somewhat-newbie. First off, I want to say that something which really feels sane and nice with this language is the "immutable by default, private by default" approach.

Now, pub is nice because it's simple and expected in modern languages. I feel that having to un-learn this in the context of Rust and sprinkle another keyword everywhere instead is a bit clumsy. Semantically, it just feels right for it to mean "this is a button that appears on the box", the box being the module: visibility "from one level up".

So, to me, using crate or another keyword of that nature instead of pub just feels itchy: if being public is the non-default, being exported out of the crate should be even more special. That is, the crate's API should be denoted in a special way.

So, I wholeheartedly agree with @epage, pub should stay the same and a pub(extern) of some kind be introduced. Parenthesized keywords really feel hairy though, so maybe it'd deserve a dedicated keyword. crate keyword would work in that sense, I can see it meaning "this an exported crate member". Or export actually, I don't know. Maybe all of my point is bikeshed though, and this all amounts to "the keywords aren't right". But pub is so common that it doesn't feel special, so it shouldn't represent something really special (the crate-exported API).

steveklabnik commented 6 years ago

I watched a talk at RustConf this weekend that used a lot of pub(crate) in its code samples, and it really really made me wish for plain old crate. I am still very pro the original plan.

epage commented 6 years ago

@steveklabnik

I watched a talk at RustConf this weekend that used a lot of pub(crate) in its code samples, and it really really made me wish for plain old crate. I am still very pro the original plan.

Is the context of this comment mostly from RustConf or does it does it take into account this thread and presuppose a disagreement with it? Earlier, I provided an alternative solution, not to pub(crate) but to the requirements driving any of the pub changes and I am hopeful it would satisfy people's needs.

See

epage commented 6 years ago

@superseed

So, I wholeheartedly agree with @epage, pub should stay the same and a pub(extern) of some kind be introduced. Parenthesized keywords really feel hairy though, so maybe it'd deserve a dedicated keyword. crate keyword would work in that sense, I can see it meaning "this an exported crate member". Or export actually, I don't know. Maybe all of my point is bikeshed though, and this all amounts to "the keywords aren't right". But pub is so common that it doesn't feel special, so it shouldn't represent something really special (the crate-exported API).

RE "Parenthesized keywords really feel hairy though"

While personally I thought they were neat when I found out about them (much better than all-or-nothing friend), my bigger concern is we don't create a parallel syntax but either embrace what we have or find an alternative solution.

On the other hand...

RE Or export actually,

I don't think adding an export contradicts my earlier comment in contrast to crate. In this context, export is could be treated as different than visibility. export would imply pub(crate). I suspect this won't have much issue in teaching.

I can go either way on this extension of my original idea.

SimonSapin commented 6 years ago

@superseed

pub […] visibility "from one level up". crate keyword would work in that sense, I can see it meaning "this an exported crate member".

I think that your understanding of the meaning of these two keywords may be the exact opposite of what is proposed here, which is that pub means public to everyone while crate means accessible from the same crate.

superseed commented 6 years ago

@epage

In the case of pub(crate), it does offer a really neat feature, and actually reads well, but looks too much like a function call in my eye. i.e. without syntactic highlighting I'd probably be really confused, and highlighting shouldn't be necessary to understand the semantics of a language.

@SimonSapin

Indeed, and I realize that's the way it's supposed to be understood, but crate -- being a noun -- feels like it is declaring either a crate (?) or a property of the crate. Emphasis declaring and not qualifying.

And public/pub is such an ubiquitous qualifier, it doesn't feel (to me!) like it should mean "this is exported out of the crate". It has this meaning of "this is visible from just outside what context I'm in", as is the case with its use in qualifying struct member visibility (and correct me if I'm wrong but I don't think the semantics are changing in that case).

rpjohnst commented 6 years ago

And public/pub is such an ubiquitous qualifier, it doesn't feel (to me!) like it should mean "this is exported out of the crate". It has this meaning of "this is visible from just outside what context I'm in", as is the case with its use in qualifying struct member visibility (and correct me if I'm wrong but I don't think the semantics are changing in that case).

pub has always meant "this is exported out of the crate"- this isn't a change, it's how it already is. The fact that so many assume otherwise is why the pub(crate) visibility level is being pushed at all.

seanmonstar commented 6 years ago

pub has always meant "this is exported out of the crate"

I think this understanding may also cause some confusion, because it's not the full picture. pub truly means "I don't care who outside of this module accesses this item/field/method". To be exported from the crate, it still requires that a path to the item also has the same pub modifiers on them.

This detail is pretty usual in many languages. It's also why I'm not keen on the unreachable_pub lint, since that is part of what is pushing this issue so much. If I have a type that is never exported at the top level, then hassling me that I marked it's methods pub just feels like noise.

superseed commented 6 years ago

@rpjohnst Is it really what it always meant? Wasn't it the chain of "visible from super" from the top of the crate which made an element exported, and not qualifying the leaf element itself as pub?

rpjohnst commented 6 years ago

No, that's not the whole story, and @seanmonstar's clarification hints at the rest. The biggest exception is reexports- you can pub use something whose parent modules are private. A weirder example is something like this, where you're allowed to use a pub item in a public interface even if its parent modules are private.

And going in the other direction, pub(crate) and lesser visibilities can't be bypassed in the same way- you can't pub use something that's not already pub, even if the pub use is not itself visible from outside the crate.

So an item's own visibility is not directly about its visibility to super, but its "upper limit" of visibility anywhere.

superseed commented 6 years ago

Oooh okay, thanks for the clarification! I had a much more naive model in mind. It makes more sense regarding previous comments about "the current meaning of pub".

nikomatsakis commented 6 years ago

We discussed briefly today in @rust-lang/lang meeting:

nikomatsakis commented 6 years ago

Answer: we parse it as a path (playground).

This seems.. probably ok to me, because I think that ::foo::bar paths will become increasingly rare.

epage commented 6 years ago

Many of us feel positive about this but there are lingering doubts about the choice of keyword, and whether it can create confusion when combined with crate::foo::bar paths

@nikomatsakis are there meeting notes or a summary for us to catch up on? Within this thread, I've not seen discussed at least one of my concerns[0] nor much discussion on the counter proposals. Maybe [0] and some of the others were discussed in the various internals threads but that is a lot to dig through.

[0] creating a parallel visibility syntax pushing pub(...) into obscurity with a feeling we should either remove or embrace pub(...)

Centril commented 6 years ago

@epage

@nikomatsakis are there meeting notes or a summary for us to catch up on?

No sorry; We did not discuss it for very long (a few minutes at most) and did not write down any meeting notes about it.

@eddyb briefly alluded to my as a shorter and more ergonomic visibility modifier.

eddyb commented 6 years ago

I think I said mine but my is even shorter, lovely! (For the record, I was half-joking in the meeting)

EDIT: if my is crate-local, can we replace pub with our? e.g.:

our struct Foo(my FooImpl);
petrochenkov commented 6 years ago

(For the record, I was half-joking in the meeting) if my is crate-local, can we replace pub with our?

Perl: making jokes a reality. https://perldoc.perl.org/functions/my.html https://perldoc.perl.org/functions/our.html

glaebhoerl commented 6 years ago

my is nice (and has come up before), the thing is that it's not any clearer than local, internal, or whatever, with respect to to what (or in its case, whose), which is the whole problem.

The awkward situation we're in is that we want to have three privacy levels -- "completely public", "completely private", and "somewhere in between" (i.e., crate-level) -- and due to backwards compatibility constraints we're stuck with the first of these necessarily being pub, the second being the implicit default, and having to come up with something new for the third. And English does not have many words which denote "neither completely global, nor completely local, but somewhere in between" with precision.

And the crate keyword is the one which does satisfy this, because it says right in the name what the actual scope is -- it's the crate. But (before you break out the "I knew it!"s), the price of this is that it's no longer evident that it's a visibility modifier. "Pub" is short for "public", that, one can intuit. But no other language has the concept of a "crate" (with that name); to have any hope of making sense of crate struct, one has to first learn about this.1 It requires us to make a further withdrawal from the "language strangeness budget", and opinions may differ about whether the balance is still positive.

Meanwhile pub(crate) solves both problems -- it tells you that it's a visibility modifier, and it tells you what the scope is -- but in exchange it's long and awkward.

So that's basically the shape of the predicament.

1 (Someone above described an interaction which went, "they asked what crate meant, I told them it's a visibility modifier, and that was the end of it". The problematic, and possibly more common, situation is when you don't happen to have a Rustacean sitting next to you.)

petrochenkov commented 6 years ago

FWIW, I'd be totally okay with our or my for crate-local items. our is even a three-letter keyword and aligns nicely with pub ~and with Perl~. Is the problem that they sound too informal (for native English speakers?)?

johnthagen commented 6 years ago

How do people feel about intern? It's one character longer than crate, but other than that I think it will be more intuitive than crate to people new to Rust and has some nice symmetry to theextern keyword.