[Stabilization] Pin APIs

withoutboats commented 5 years ago

@rfcbot fcp merge Feature name: pin Stabilization target: 1.32.0 Tracking issue: #49150 Related RFCs: rust-lang/rfcs#2349

This is a proposal to stabilize the pin library feature, making the "pinning" APIs for manipulating pinned memory usable on stable.

(I've tried to write this proposal as a comprehensive "stabilization report.")

Stabilized feature or APIs

`[std|core]::pin::Pin`

This stabilizes the Pin type in the pin submodule of std/core. Pin is a fundamental, transparent wrapper around a generic type P, which is intended to be a pointer type (for example, Pin<&mut T> and Pin<Box<T>> are both valid, intended constructs). The Pin wrapper modifies the pointer to "pin" the memory it refers to in place, preventing the user from moving objects out of that memory.

The usual way to use the Pin type is to construct a pinned variant of some kind of owning pointer (Box, Rc, etc). The std library owning pointers all provide a pinned constructor which returns this. Then, to manipulate the value inside, all of these pointers provide a way to degrade toward Pin<&T> and Pin<&mut T>. Pinned pointers can deref, giving you back &T, but cannot safely mutably deref: this is only possible using the unsafe get_mut function.

As a result, anyone mutating data through a pin will be required to uphold the invariant that they never move out of that data. This allows other code to safely assume that the data is never moved, allowing it to contain (for example) self references.

The Pin type will have these stabilized APIs:

`impl Pin where P: Deref, P::Target: Unpin`

fn new(pointer: P) -> Pin

`impl Pin where P: Deref`

unsafe fn new_unchecked(pointer: P) -> Pin
fn as_ref(&self) -> Pin<&P::Target>

`impl Pin where P: DerefMut`

fn as_mut(&mut self) -> Pin<&mut P::Target>
fn set(&mut self, P::Target);

`impl<'a, T: ?Sized> Pin<&'a T>`

unsafe fn map_unchecked<U, F: FnOnce(&T) -> &U>(self, f: F) -> Pin<&'a U>
fn get_ref(self) -> &'a T

`impl<'a, T: ?Sized> Pin<&'a mut T>`

fn into_ref(self) -> Pin<&'a T>
unsafe fn get_unchecked_mut(self) -> &'a mut T
unsafe fn map_unchecked_mut<U, F: FnOnce(&mut T) -> &mut U>(self, f: F) -> Pin<&'a mut U>

`impl<'a, T: ?Sized> Pin<&'a mut T> where T: Unpin`

fn get_mut(self) -> &'a mut T

Trait impls

Most of the trait impls on Pin are fairly rote, these two are important to its operation:

impl<P: Deref> Deref for Pin { type Target = P::Target }
impl<P: DerefMut> DerefMut for Pin where P::Target: Unpin { }

`std::marker::Unpin`

Unpin is a safe auto trait which opts out of the guarantees of pinning. If the target of a pinned pointer implements Unpin, it is safe to mutably dereference to it. Unpin types do not have any guarantees that they will not be moved out of a Pin.

This makes it as ergonomic to deal with a pinned reference to something that does not contain self-references as it would be to deal with a non-pinned reference. The guarantees of Pin only matter for special case types like self-referential structures: those types do not implement Unpin, so they have the restrictions of the Pin type.

Notable implementations of Unpin in std:

impl<'a, T: ?Sized> Unpin for &'a T
impl<'a, T: ?Sized> Unpin for &'a mut T
impl<T: ?Sized> Unpin for Box<T>
impl<T: ?Sized> Unpin for Rc<T>
impl<T: ?Sized> Unpin for Arc<T>

These codify the notion that pinnedness is not transitive across pointers. That is, a Pin<&T> only pins the actual memory block represented by T in a place. Users have occassionally been confused by this and expected that a type like Pin<&mut Box<T>> pins the data of T in place, but it only pins the memory the pinned reference actually refers to: in this case, the Box's representation, which a pointer into the heap.

`std::marker::Pinned`

The Pinned type is a ZST which does not implement Unpin; it allows you to supress the auto-implementation of Unpin on stable, where !Unpin impls would not be stable yet.

Smart pointer constructors

Constructors are added to the std smart pointers to create pinned references:

Box::pinned(data: T) -> Pin<Box<T>>
Rc::pinned(data: T) -> Pin<Rc<T>>
Arc::pinned(data: T) -> Pin<Arc<T>>

Notes on pinning & safety

Over the last 9 months the pinning APIs have gone through several iterations as we have investigated their expressive power and also the soundness of their guarantees. I would now say confidently that the pinning APIs stabilized here are sound and close enough to the local maxima in ergonomics and expressiveness; that is, ready for stabilization.

One of the trickier issues of pinning is determining when it is safe to perform a pin projection: that is, to go from a Pin<P<Target = Foo>> to a Pin<P<Target = Bar>>, where Bar is a field of Foo. Fortunately, we have been able to codify a set of rules which can help users determine if such a projection is safe:

It is only safe to pin project if (Foo: Unpin) implies (Bar: Unpin): that is, if it is never the case that Foo (the containing type) is Unpin while Bar (the projected type) is not Unpin.
It is only safe if Bar is never moved during the destruction of Foo, meaning that either Foo has no destructor, or the destructor is carefully checked to make sure that it never moves out of the field being projected to.
It is only safe if Foo (the containing type) is not repr(packed), because this causes fields to be moved around to realign them.

Additionally, the std APIs provide no safe way to pin objects to the stack. This is because there is no way to implement that safely using a function API. However, users can unsafely pin things to the stack by guaranteeing that they never move the object again after creating the pinned reference.

The pin-utils crate on crates.io contains macros to assist with both stack pinning and pin projection. The stack pinning macro safely pins objects to the stack using a trick involving shadowing, whereas a macro for projection exists which is unsafe, but avoids you having to write the projection boilerplate in which you could possibly introduce other incorrect unsafe code.

Implementation changes prior to stabilization

[ ] Export Unpin from the prelude, remove pin::Unpin re-export

As a general rule, we don't re-export things from multiple places in std unless one is a supermodule of the real definition (e.g. shortening std::collections::hash_map::HashMap to std::collections::HashMap). For this reason, the re-export of std::marker::Unpin from std::pin::Unpin is out of place.

At the same time, other important marker traits like Send and Sync are included in the prelude. So instead of re-exporting Unpin from the pin module, by putting in the prelude we make it unnecessary to import std::marker::Unpin, the same reason it was put into pin.

[ ] Change associated functions to methods

Currently, a lot of the associated function of Pin do not use method syntax. In theory, this is to avoid conflicting with derefable inner methods. However, this rule has not been applied consistently, and in our experience has mostly just made things more inconvenient. Pinned pointers only implement immutable deref, not mutable deref or deref by value, limiting the ability to deref anyway. Moreover, many of these names are fairly unique (e.g. map_unchecked) and unlikely to conflict.

Instead, we prefer to give the Pin methods their due precedence; users who need to access an interior method always can using UFCS, just as they would be required to to access the Pin methods if we did not use method syntax.

[ ] Rename get_mut_unchecked to get_unchecked_mut

The current ordering is inconsistent with other uses in the standard library.

[ ] Remove impl Unpin for Pin

This impl is not justified by our standard justification for unpin impls: there is no pointer direction between Pin and P. Its usefulness is covered by the impls for pointers themselves.

This futures impl will need to change to add a P: Unpin bound.

[ ] Mark Pin as repr(transparent)

Pin should be a transparent wrapper around the pointer inside of it, with the same representation.

Connected features and larger milestones

The pin APIs are important to safely manipulating sections of memory which can be guaranteed not to be moved out. If the objects in that memory do not implement Unpin, their address will never change. This is necessary for creating self-referential generators and asynchronous functions. As a result, the Pin type appears in the standard library future APIs and will soon appear in the APIs for generators as well (#55704).

Stabilizing the Pin type and its APIs is a necessary precursor to stabilizing the Future APIs, which is itself a necessary precursor to stabilizing the async/await syntax and moving the entire futures 0.3 async IO ecosystem onto stable Rust.

cc @cramertj @RalfJung

Keruspe commented 5 years ago

My 2 cts for the bikeshedding:

If I understand correctly, what Unpin realy means is "Pin has no effect on me".

What about something like BypassPin or IgnorePin?

RalfJung commented 5 years ago

So let's assume I want to create a type Woof for which it is also always possible to transition from all of Woof's states while in the pinned type-state to the not-pinned type-state, but it isn't trivial to do so and can therefor not implement Unpin, would it be allowed for Woof to have a function like fn unpin(self: Pin<Box>) -> Box?

Yes, that should be possible. For example, if Woof is an element of an intrusive linked list, it could provide a function to remove the element from the list, and remove the Pin at the same time (arguing that for non-enqueued Woofs, the two typestates are equivalent, so we can de-pin it).

Nemo157 commented 5 years ago

Relocate has the same issue as RePin to me, it could imply an operation allowing to move a value from one pinned location to another, not unpinning the value completely. It’s a less strong connotation than RePin has, but still seems a little confusing.

RalfJung commented 5 years ago

it could imply an operation allowing to move a value from one pinned location to another, not unpinning the value completely.

While that's not exactly what this trait is about, it is not entirely wrong either: For most use-cases, moving a value from one pinned location to another is just as catastrophic as using it freely. Actually, given that anyone can create a Pin<Box<T>>, I don't even see why you are making a fundamental distinction here.

Nemo157 commented 5 years ago

For most use-cases, moving a value from one pinned location to another is just as catastrophic as using it freely.

Yes, which is why having a trait adding that operation to a type could be useful (this could not be a marker trait like Unpin as it may need to do things like update internal references). I’m not proposing we add something like this now, just that it’s something I could see being provided in the future (either in std or third party) that could end up with a confusing overlap of names.

I don’t know of any strong use cases for it now, but I have thought about using something like it with pinned collections.

alexcrichton commented 5 years ago

Ok so here's some thinking. I was initially wondering if we could have a name like PinInert or PinNoGuarantees as that's also what the description is, but thinking about that what I really want is a trait describing an action as opposed to a property, or at least it makes it much easier to think about in my head.

One problem with Unpin (I'm not sure why) is that I can't seem to get it through my head that the intended meaning is "the action of removing from a pin, unpinning, is safe to do". The trait as-is conveys action "the act of unpinning" but I can't seem to quite understand that when I read it. It feels weird to have Pin<T: Unpin> because if it's "unpin" why is it in a Pin?

I wonder if a name like CanUnpin could work? A lot of this isn't about a hard guarantee one way or another (implementing Unpin doesn't mean that you will remove it from a pin, it just means you can remove it from a pin). How does that sound? Is CanUnpin readable enough to others? Short enough?

(having the prefix of Can conveys the verb to me much more easily as well, and it says you can remove it from a pin but you're not necessarily always going to).

As another unrelated tangent, one thing I forgot to bring up earlier (sorry about that!) is that I don't think that all of the methods above should necessarily be inherent methods. We've already had loads and loads of bugs around inference and clashing methods, and adding very common names like as_ref and as_mut on types which also implement Deref seems like a problem waiting to happen.

Do these operations happen commonly enough to justify the risky location to put them? (inherently) Or are they used rarely enough that the safe route of associated functions can be stomached?

anp commented 5 years ago

Since I apparently have skin in this game now: CanUnpin seems very close in spirit to me to Unpinnable or something similar, and my impression has always been that the community frowns upon those kinds of modifiers in trait names, since most traits describe the action that the type can take. The impl itself is an implied "can" or "-able" in that sense. Whatever is decided (and the name doesn't matter to the extent stated here IM-not-so-HO -- very few users will likely have to type this), I would encourage everyone to feel some urgency about resolving the questions here. I'm excited for this stabilization, and I know that many many people are!

ghost commented 5 years ago

@alexcrichton

One problem with Unpin (I'm not sure why) is that I can't seem to get it through my head that the intended meaning is "the action of removing from a pin, unpinning, is safe to do".

What if you think of T: Unpin as "T is immune to Pin"? Then if you have Pin<T: Unpin> it just means we've got a value inside a Pin which is immune to pinning so pinning is here effectively moot.

Or in other words: Unpin neutralizes Pin. To be honest, after so much discussion I've kind of internalized this and now it makes sense. 😆

A lot of this isn't about a hard guarantee one way or another (implementing Unpin doesn't mean that you will remove it from a pin, it just means you can remove it from a pin).

A counterpoint to this is the Send trait. It doesn't mean that you will send the value, it just means you can send it.

alexcrichton commented 5 years ago

@anp I agree that CanUnpin isn't a great name, I'm attempting to do my damndest to figure out a better name! It seems few still feel this needs to be renamed though as all suggestions seem to get shot down for basically the same reasons I feel like Unpin needs renaming in the first place.

Also as a reminder any stabilization rides the trains like all other changes, and with a release next week this definitely won't make it into that release and will next be a candidate for the next release. That means we have 7 weeks, nearly two months, to land all this to ensure it gets in ASAP. While I agree urgency is necessary it's just "let's not forget about this urgency", not "let's resolve this before Monday" urgency.

@stjepang I, too, have grown used to Unpin now after thinking about it so much! All alternative names at this point seem quite lackluster, so I'm getting to the point where I would settle for better documentation. I would ideally like to find someone who doesn't know much about the Pin apis to read said documentation afterwards, though, to double check it's sufficient for learning.

alexcrichton commented 5 years ago

I'd also like to go ahead and formally block stabilization on improved documentation, as it feels especially critical for this issue.

@rfcbot concern improving-the-documentation

Concretely what I feel needs better documentation is:

More prose on each method about why it is safe and/or why it is unsafe. For example the contract of P's DerefMut implementation "behaving reasonably" isn't documented on Pin::new_unchecked.
Each unsafe function should have a code example of why it is unsafe, or at least a clear explanation of a sequence of steps that can go wrong.
I feel the module docs can go into more detail beyond simply what a Pin is and what it means. I think they'd benefit from information like "how this isn't general immovable types but a form of that" or "how is Pin used in practice, for example with futures".
I'd like to see an example with generics and Unpin in either the documentation or code examples. For example it sounds like many combinators in futures have to handle this, and similarly some combinators specifically require this. Some example use cases of how one works with generics and how they play out might help understanding Unpin quite a bit.

I'm also curious if others have concrete actionable items to add to the documentation too!

alexcrichton commented 5 years ago

Next I'd also like to formally block on the self-question with as_ref, but I suspect this will be quick to resolve. (again sorry for not remembering to bring this up sooner)

@rfcbot concern self-methods

This proposes as_ref, as_mut, get_ref, get_mut, into_ref, and set to all be methods on the Pin type. The proposal mentions that we don't do this for smart pointers because of method clashes, but cites that experience shows today's implemented Pin API isn't consistent and it ends up often just getting in the way.

These method names, however, are very short and sweet names and are quite common to find throughout the ecosystem. The libs team has run into a never ending series of bugs in the past where we add trait implementations and such and it causes clashes with existing trait methods in various crates. These methods seems especially high risk because of their names. Additionally, it's unclear whether we could ever add more methods to Pin in the future as even if the names now end up working out any future name added has a high risk of collision.

I'd like to just be sure that we rethink this divergence from convention, I'm personally particularly worried about the consequences and I think it'd be great if we could find a middle-ground which would avoid these hazards.

alexcrichton commented 5 years ago

And while I'm at it, one last thing I've noticed, and again I think this'll be pretty quick to resolve

@rfcbot concern box-pinnned-vs-box-pin

Has the name Box::pin been considered for Box::pinned? With the existence of Pinned and/or PhantomPinned it seems like it'd be neat if the name could match the return type!

glaebhoerl commented 5 years ago

@alexcrichton Has anything in particular persuaded you against MoveFromPin as an option? I see you were favorable towards it earlier (and multiple people seemed to like it as well). For the record, the direct objections I could remember, or quickly find by Ctrl+F-ing, are that "names that are whole phrases ... would be very unidiomatic" (@withoutboats) and that it's "far too wordy" (@cramertj).

I have to admit that motivations in the vein of "I have grown used to ... now after thinking about it so much" make me rather uneasy. Humans can get used to and post-hoc-rationalize essentially anything. No matter what the name is, it would be a major surprise if we didn't eventually end up getting used to it. (This is the very reason why we choose obviously-suboptimal names like existential type as temporaries in other cases, to try to avoid them becoming permanent.)

Accidentally privileging the perspectives of experienced users (which we all are!) over others who will be coming at things with fresher brains is the kind of mistake I think we need to be ever-vigilant against making, as hard as it is.

Kimundi commented 5 years ago

While it certainly isn't the style of the std lib, I also feel like CanUnpin somehow makes it way more clear how that particular piece fits together with the other ones.

valff commented 5 years ago

It is impossible to invent a name which describes the meaning of Unpin to a beginner without reading documentation. Send and Sync are also hard to understand for a beginner, but they look nice and concise, as well as Unpin. If you don't know what these names mean, you must read their docs.

Pauan commented 5 years ago

@valff Indeed that is correct, however there are quite a few people who have read the docs and understand how pinning works, yet the Unpin name still causes them significant mental difficulty, whereas the other names cause less.

alexcrichton commented 5 years ago

@glaebhoerl oh sorry to clarify I'm personally more of a fan of MoveFromPin or CanUnpin than the current Unpin. Just trying to help move to a conclusion!

HeroicKatora commented 5 years ago

I tried understanding the current proposal, maybe this sort of progress report can shine some additional light on naming as well.

Extending the guarantees of Pin to a stable address until Drop puts additional requirements on the creator of a Pin. I would have found it useful if those necessary preconditions for invoking unsafe fn new_unchecked(pointer: P) -> Pin were mentioned directly. Understanding how a Pin can be created helps hugely in understanding their concept, imo.
The summary mentioned in the tracking issue, prefaced with we agreed that we are ready to stabilize them with no major API changes., contains a lot of old api. At the same time, it also contains a lot of insight about Drop and pin projection that is relevant. It also contains an explicit formulation of the additional guarantee referred to in bullet point one that is not found in this report. This mix of outdated and fresh information was a bit confusing, consider moving all of the information into this issue or indicating outdate paragraphs.
Since the additional guarantee is referred to as a slight extension, I expected there to be some kind of way to opt out of it. Since pinning once carries a type state on that pointer until it is dropped, some way to go back from this type state to a 'normal' state. In fact, that was what I thought Unpin to do at first but I think Unpin is actually a bit stronger. It says the the pinned state can be freely turned into an unpinned state, at will. It's fine that the current interface is minimal in this regard but Unpin could be confused with an operation on the type that removes some inner invariant that requires the pin state (like a soft version of Drop).

Clarification: I personally favour MoveFromPin over the other names but it is arguably artificial and clunky.

The major actionable concrete actionable item that comes to mind, besides the documentation improvements already requested by @alexcrichton , would be explaining the reasoning behind the field rule for pin projection in map_unchecked and map_unchecked_mut. Something along the lines of:

A pin promises to not move the referred data until it is dropped. Since the fields of a struct are dropped after their containing struct, their pin state extends beyond the Drop implementation of the structs itself. Projecting to a field promises not to move from it within the structs destructor.

HeroicKatora commented 5 years ago

What about the name ElidePin for the auto derived marker trait?

It would capture that the type can always be treated as if or behaves as if it were unpinned. And !ElidePin seems to be quite clear as well, although that might be subjective.

The standard marker doesn't have a perfect name for understanding pinning as well. Pinned seems to evoke the notion that the containing struct itself is pinned but pinning applies to pointers and only after an explicit action. This is not quite as critical but not insignificant either, especially since it could be the type that is first encountered in an implementation for a self-referential type.

cramertj commented 5 years ago

My general opposition to MoveFromUnpin/RemoveFromUnpin is just that they're far too wordy and would regress the ergonomics of manual Future implementations. CanUnpin/DePin both seem fine to me-- I wish it was more clear that Unpin follows the pattern of Read/Write/etc. but it seems that's not intuitive to folks so I'll +1 anything that makes this clearer without looking like syntactic salt.

jeekobu commented 5 years ago

I agree NotWhateverUnpinBecomes is probably the best name for Pinned. That said, Adhere means both to heed and to stick to. :slightly_smiling_face:

wmanley commented 5 years ago

CanUnpin/DePin both seem fine to me-- I wish it was more clear that Unpin follows the pattern of Read/Write/etc

I think one of the things that makes Unpin hard, unlike Read is that it is a marker trait. Read is easy to understand because there is a method Read::read - everything you need to know is right there in the trait. If x is Read I understand that I can call x.read() - similarly write for Write, etc. It's harder to explain that X implementing Unpin means that Pin<Ptr<X>> implements DerefMut - which means that you can treat it as if it were just X.

cramertj commented 5 years ago

Read is easy to understand because there is a method Read::read

If only we could add always-applicable methods in auto trait definitions + GATs, we could have Unpin::unpin-- fn unpin<P: DerefFamily>(self: Pin<P::Deref<Self>>) -> P::Deref<Self>). .....on second thought, I don't think that's going to make anyone less confused ;)

(on a more serious note, I'd support Pin::unpin for going from Pin<P<T>> to P<T>)

HeroicKatora commented 5 years ago

I'd support Pin::unpin for going from Pin to P

This confuses two terms in my head, just like the current name itself. unpin sounds very much like reversing the guarantees of the type state, for a comparison that would be as if there was a method fn unborrow(&'_ T) or fn unmove(??). Since pin provides guarantees until some memory representing a type is Drop::droped, we don't really reverse the state, the type merely guarantees that all other representations uphold equivalent guarantees and we can thus ignore this. This is also the main difference I see between marker traits and something like io::Read. One enables operations for the compiler or language, while the other enables operations for the programmer.

Furthermore, it is a major point that this cannot currently be accurately represented just using correct typing. To call the operation unpin makes it sound like there was a converse function pin. It being a function also slightly wrongly hints that such an unpin operation is somehow bound to computational work, for example into_pointer() would be bettter in this regard by also following a more established naming convention.

Lastly, I think there's potential for types specifically having an unpin function with computational work. A type with very special inner invariants might be able to 'fix' its inner state in such a way that it can offer an interface fn unpin(self: Pin<&'a mut T>) -> &'a mut, where it willingly forfeits all guarantees of Pin for some lifetime 'a. In that case, both of the above points no longer apply. Such a function could be envisioned as if it had equivalent effect to dropping and reconstructing in the same memory location (thus actually removing the type state). And it may involve computation, e.g. by moving some self references into dynamic allocations.

It would be unfortunate if a confusing name would also make it harder for library designers and implementors to choose non-confusing names, by conflating these two ideas.

Matthias247 commented 5 years ago

My general opposition to MoveFromUnpin/RemoveFromUnpin is just that they're far too wordy and would regress the ergonomics of manual Future implementations.

I don't think that's true, especially for MoveFromPin, which seems reasonable to type for me - and with any kind of smart or dumb autocompletion the problem is mostly nonexistent anyway.

One important part of ergonomics should be also readability and understandability of code. Rust already earned a bit of criticism in the past for abbreviating things aggressively (fn, mut, etc), which makes some code harder to read. For things that for a concept that is even more complicated to grasp and which fulfills a niche purpose for most users, using a more verbose and descriptive name should be totally acceptable.

alexcrichton commented 5 years ago

@rfcbot resolve naming-of-Unpin

Ok I've been stewing for awhile now and have also talked with @withoutboats in person about this. I've come around that I'm now personally comfortable at least with the name Unpin as the marker trait. As a result, I'm going to remove a blocking objection (although if others on the libs team feel differently, please let us know!)

The main reason that I've come around to Unpin is basically what @cramertj already said above, it's the most idiomatic name for this trait that we've been able to come up with. All other trait names I've seen proposed don't meet the idiomatic bar that Unpin lives up to (or at least in my opinion).

While I still believe there are better names for "I just saw the name of this trait and I want to know what it does", I don't think that's the right problem to solve here. It's clear to me at this point that we can't, at this time at least, figure out a different name for this trait which is also idiomatic, but rather we're switching around on trait names that solve a different problem. Understanding Unpin is just like Send and Sync, the documentation needs to be read to understand fully what's going on.

As another clarifying point, I listed a few "blocking objections" but they're moreso TODO items than blocking objections per se. I just don't have a great way of otherwise navigating such a long thread! In that vein I think it's fine for a stabilization PR to be posted at any time now with FCP lapsed for a week or so now. The final points about self/pin/pinned can be briefly discussed there (if even necessary, they could be left as the proposal above).

Documentation I think especially is not a prerequisite for stabilization in this case. We've got a full cycle (6 weeks) to add docs before these types hit stable and we've got even longer to shore up the documentation here before the full async/await story hits stable. That's loads of time to improve what's there now, and what's there now is already quite useful!

jaredr commented 5 years ago

What does "idiomatic" even mean here? What's unidiomatic about Reloc[ate], Escape, Evade, Pluck, or Roam? They're all verbs, and none of them can be mistaken for having the double-negative problem.

Pauan commented 5 years ago

@alexcrichton Is there a reason why Unpin is considered more idiomatic than Depin or DePin?

I think Depin is a very solid alternative, and it doesn't cause me the same mental difficulties as Unpin.

jimmycuadra commented 5 years ago

One simple reason might be that depin isn't a word and unpin is. Personally, I don't have any trouble understanding Unpin. I think it fits with the naming of the other marker traits.

Pauan commented 5 years ago

@jimmycuadra There's a lot of names in Rust which aren't "real" words, including in the stdlib.

I would be surprised if that was considered a significant reason to choose one name over another.

withoutboats commented 5 years ago

@Pauan It is a significant reason. As a native English speaker, Depin sounds to me like we forgot that the word for unpinning exists and tried to make one up. It seems glaringly wrong to me gramatically: the English word for "depinning" is "unpinning."

A good analogy would be if we had APIs that said "delock" instead of "unlock".

alexcrichton commented 5 years ago

@jaredr by "idiomatic" I mean following the established conventions of the standard library already like the (previously mentioned) Read and Write traits. We have a convention of not-wordy-names, verbs, short where possible, and appropriate for the situation.

Names like you've suggested are all possible, but Unpin (or so I feel at least) is the most appropriate for this particular action ("you can unpin this type" guarantee). The others, while loose synonyms of Unpin, I think are largely only renaming away from the word "unpin" and sometimes don't convey the same connection with Pin as Unpin does.

@Pauan I agree with @withoutboats that Depin doesn't sound like it jives as well as Unpin.

Centril commented 5 years ago

@aturon noted in https://github.com/rust-lang/rfcs/pull/2592#issuecomment-438873636 that:

The use of Pin is no more of an implementation detail than the choice of &self versus &mut self; in the long run, it's likely that Pin will itself be a separate mode of reference with language support. In all these cases, what's being conveyed in the signature is the permissions being granted to the callee, with Pin forbidding moving out of the reference.

Ostensibly this refers to a native &pin type or something... but idk how this squares with Pin<&mut T> and such. In my conversations with @withoutboats and in particular @cramertj they were not at all sure about the idea of a separate mode of reference with language support and how we could get there from Pin<T>.

Before stabilizing pin, it would be prudent to reconcile these views to ensure that we are on the same page wrt. this critical piece of infrastructure; so I'd mainly like for Aaron to expand on this and for boats and Taylor to then weigh in.

crlf0710 commented 5 years ago

The others, while loose synonyms of Unpin, I think are largely only renaming away from the word "unpin" and sometimes don't convey the same connection with Pin as Unpin does.

@alexcrichton this is actually a good thing, i think? Like Send to move/copy (=) operation, Sync to borrow (&) operation. Maybe such connection is actually causing more confusion?

alexcrichton commented 5 years ago

@crlf0710 it may! I'm not sure I would agree with such a connection though myself. Send and Sync are more about what they're enabling ("send"ing types to other threads and "sync"hronizing access across threads), and when naming them we didn't try to avoid naming them close to one operation or another

crlf0710 commented 5 years ago

@alexcrichton exactly! so maybe this trait should also be about what it is enabling. ("~~moving~~(A verb here) out of a Pin). I'm not native english speaker, but i still think "unpining" out of a pin is a little...weird?

withoutboats commented 5 years ago

@crlf0710 But what the trait enables is not <moving> out of a Pin, its <moving out of a Pin>. The problem with move and synonyms for move is that they imply the trait controls the ability of the type to move at all, which it does not do. The connection to Pin is vital to understanding what the trait actually does.

So the most idiomatic name for this trait would be a verb which means "to move out of a pin," for which the word "Unpin" is the most obvious to me by a lot. Here's Wiktionary's definition of "unpin":

To unfasten by removing a pin.

(transitive, computing, graphical user interface) To detach (an icon, application, etc.) from the place where it was previously pinned.

crlf0710 commented 5 years ago

@withoutboats thanks for explaining! Actually i think i can accept this Unpin name or any name rust team finally decides on, i don't want to block the stabilization of Pin types at all, and i absolutely understands the concerns about "moving" etc.

It just feels there's a very slightest "repetition" somewhere. and i have to persuade my self: it doesn't mean "unpinnable", but the opposite. I guess after some time i'll get used to it if that's the final decision.

At the same time please allow me to make my last suggestion: actually I think the Detach verb above mentioned is also quite nice, though a little too general. As i said, i'm not native speaker, so i can't speak for others. Please just see it as a little idea.

qnighy commented 5 years ago

I was thinking about safe disjoint pinned projections, and came up with the idea that may enable it. This is a macro to simulate matching against pinned mutable lvalues. With this macro we can write

pin_proj! { let MyStruct { field1, field2 } = a_pinned_mutable_reference; }

to decompose a Pin<&mut MyStruct> into pinned mutable references to the fields.

To make this safe and usable, we'll need two other things:

The Kite type to mark "always-Unpin" fields
Make Unpin unsafe

The latter is needed for the safety of the projection. Without this, we can define "Kite with pinned projections" safely, which is actually wrong.

With this in mind, I propose to make Unpin unsafe before stabilization to leave a room for other useful operations to be safe.

withoutboats commented 5 years ago

@qnighy Its possible for a type to violate pinning guarantees in its Drop implementation, which makes Drop equivalently powerful to Unpin. Making Unpin unsafe does not make any additional code safe, because Drop is also safe, and that can't be changed. We've talked about this a lot on the tracking issue, and its been mentioned on this thread as well.

Fundamentally, pin projections cannot be made safe without the ability to assert certain negative bounds that can't be expressed in Rust today.

@crlf0710

At the same time please allow me to make my last suggestion: actually I think the Detach verb above mentioned is also quite nice, though a little too general. As i said, i'm not native speaker, so i can't speak for others. Please just see it as a little idea.

Detach seems a lot better than names like Move and Relocate, but it seems to conflict with other possible uses of the detach metaphor (similar to how Escape could conflict with "escape analysis" etc). There's only so many metaphors we have about how data can relate to other data in computer science, which makes me think of another new advantage of Unpin: by cleaving tightly to the "pin" metaphor, it doesn't occupy space for future metaphoric language we may need to use for a different purpose, the way names like Detach or Escape or Relocate could.

Centril commented 5 years ago

@withoutboats I have no particular preference for any name or much investment in this bikeshed... but I'm curious; what other purpose would Detach fit (if you speculate...)?

withoutboats commented 5 years ago

@Centril I don't have a clear use case in mind, but I could imagine some use case related to destructuring for example.

Centril commented 5 years ago

@withoutboats Yeah that makes sense; cheers!

HeroicKatora commented 5 years ago

@withoutboats I'm not sure if the Wiktionary entry is the best motivation to justify the naming of Unpin to enable move from a pin. The physical representation metaphors suffer from the fact that moving in Rust is not taking away an object in the world. To be more precise, 'upinning' a pinned pointer does not give me control over the referred to memory, it's allocation and validity as an object representation of T are still required and guaranteed by the pointer semantics. Hence unpinning does not give a fully controlled representation of T, as unboxing would. And, a dictionary entry defining unpinning in terms of moving is in fact part of the confusion more than the justification of such a name.

In the proposed API, none of the methods has such an effect (and it's not in the scope of pin either). I don't see a safe way for example to transition Pin<Box<T>> to Box<T> (and by extension to T) and nor am I sure if Unpin should have such strong possibilities. I'm not exactly sure where all the difference would be. But as far as I understood, Pin applies to some memory location while Unpin guarantees that nothing in Ts representation relies on pin guarantees. However, would that be the same as being able to take out of and forget the memory entirely? I think not. I'll think of a more concrete example.

Edit: More concretely, even if T is Unpin, can I rely on some instance of T::drop(&mut) being called on the memory that was pinned, before that memory is deallocated? As far as I can tell from the formalism, the answer should be yes, but the name Unpin communicates to me the opposite.

Edit 2: Rc allows one to observe Pin<&T> but for T: Unpin still not have drop called on the original memory location. Keep a reference alive outside the pin, then after the pin was dropped you can move out with Rc::try_unwrap. https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=dec6f6c6d2c0903d87a4a9cefe50a0ca That effectively answers the question through existing mechanisms but does it work as intended?

wmanley commented 5 years ago

Perhaps IgnorePin? If T is IgnorePin you can treat Pin<Box<T>> as &mut T essentially ignoring the presence of Pin (AIUI also ignoring the Box). To ignore is a verb, I guess that IgnorePin isn't but I'm not sure what it is. IgnorePin is descriptive of what it allows, it's not descriptive of the constraints placed upon the type, but neither is Unpin.

HeroicKatora commented 5 years ago

@wmanley I had a very similar idea, ElidePin, in some comment above although back then I had not been able to express concretely why that one felt more precise. But I concur that 'one verb' is the style guide for Rust marker traits. Even though it also allows a more natural negation in the form of !ElidePin/!IgnorePin, it is not optimal.

HeroicKatora commented 5 years ago

@withoutboats Follow up question: Since pin seems specified in terms of the underlying memory, how does Pin interact with ZST? Due to Pinned being a ZST, even a ZST may be either Unpin or not. I would intuitively say that its memory representation is never formally invalidated, thus T::drop(&mut self) need never be called, quite similar to how one could build a pin out of &mut 'static _ memory for example from a leaked Box. At the same time, that may all be wrong and I see how another interpretation would be possible. I feel like these settings deserve attention in the documentation.

tikue commented 5 years ago

Is it safe to create a Pin<P<T>> with a T: !Unpin and then immediately unpin the value, if it's guaranteed that nothing observed the pinning? Is it only undefined behavior if the pinned value is moved after it has had a poll-like method called on it?

cramertj commented 5 years ago

@HeroicKatora Unfortunately, it's possible today to implement Drop for a type that could be !Unpin due to generics, e.g.:

struct S<T>(T); // `!Unpin` if `T: !Unpin`
impl<T> Drop for S<T> { ... }

HeroicKatora commented 5 years ago

@cramertj Thank you, just realized that as well.

rust-lang / rust