Closed withoutboats closed 5 years ago
My 2 cts for the bikeshedding:
If I understand correctly, what Unpin
realy means is "Pin
has no effect on me".
What about something like BypassPin
or IgnorePin
?
So let's assume I want to create a type Woof for which it is also always possible to transition from all of Woof's states while in the pinned type-state to the not-pinned type-state, but it isn't trivial to do so and can therefor not implement Unpin, would it be allowed for Woof to have a function like fn unpin(self: Pin<Box
>) -> Box ?
Yes, that should be possible. For example, if Woof
is an element of an intrusive linked list, it could provide a function to remove the element from the list, and remove the Pin
at the same time (arguing that for non-enqueued Woof
s, the two typestates are equivalent, so we can de-pin it).
Relocate
has the same issue as RePin
to me, it could imply an operation allowing to move a value from one pinned location to another, not unpinning the value completely. It’s a less strong connotation than RePin
has, but still seems a little confusing.
it could imply an operation allowing to move a value from one pinned location to another, not unpinning the value completely.
While that's not exactly what this trait is about, it is not entirely wrong either: For most use-cases, moving a value from one pinned location to another is just as catastrophic as using it freely. Actually, given that anyone can create a Pin<Box<T>>
, I don't even see why you are making a fundamental distinction here.
For most use-cases, moving a value from one pinned location to another is just as catastrophic as using it freely.
Yes, which is why having a trait adding that operation to a type could be useful (this could not be a marker trait like Unpin
as it may need to do things like update internal references). I’m not proposing we add something like this now, just that it’s something I could see being provided in the future (either in std
or third party) that could end up with a confusing overlap of names.
I don’t know of any strong use cases for it now, but I have thought about using something like it with pinned collections.
Ok so here's some thinking. I was initially wondering if we could have a name like PinInert
or PinNoGuarantees
as that's also what the description is, but thinking about that what I really want is a trait describing an action as opposed to a property, or at least it makes it much easier to think about in my head.
One problem with Unpin
(I'm not sure why) is that I can't seem to get it through my head that the intended meaning is "the action of removing from a pin, unpinning, is safe to do". The trait as-is conveys action "the act of unpinning" but I can't seem to quite understand that when I read it. It feels weird to have Pin<T: Unpin>
because if it's "unpin" why is it in a Pin?
I wonder if a name like CanUnpin
could work? A lot of this isn't about a hard guarantee one way or another (implementing Unpin
doesn't mean that you will remove it from a pin, it just means you can remove it from a pin). How does that sound? Is CanUnpin
readable enough to others? Short enough?
(having the prefix of Can
conveys the verb to me much more easily as well, and it says you can remove it from a pin but you're not necessarily always going to).
As another unrelated tangent, one thing I forgot to bring up earlier (sorry about that!) is that I don't think that all of the methods above should necessarily be inherent methods. We've already had loads and loads of bugs around inference and clashing methods, and adding very common names like as_ref
and as_mut
on types which also implement Deref
seems like a problem waiting to happen.
Do these operations happen commonly enough to justify the risky location to put them? (inherently) Or are they used rarely enough that the safe route of associated functions can be stomached?
Since I apparently have skin in this game now: CanUnpin
seems very close in spirit to me to Unpinnable
or something similar, and my impression has always been that the community frowns upon those kinds of modifiers in trait names, since most traits describe the action that the type can take. The impl itself is an implied "can" or "-able" in that sense. Whatever is decided (and the name doesn't matter to the extent stated here IM-not-so-HO -- very few users will likely have to type this), I would encourage everyone to feel some urgency about resolving the questions here. I'm excited for this stabilization, and I know that many many people are!
@alexcrichton
One problem with
Unpin
(I'm not sure why) is that I can't seem to get it through my head that the intended meaning is "the action of removing from a pin, unpinning, is safe to do".
What if you think of T: Unpin
as "T
is immune to Pin
"? Then if you have Pin<T: Unpin>
it just means we've got a value inside a Pin
which is immune to pinning so pinning is here effectively moot.
Or in other words: Unpin
neutralizes Pin
. To be honest, after so much discussion I've kind of internalized this and now it makes sense. 😆
A lot of this isn't about a hard guarantee one way or another (implementing
Unpin
doesn't mean that you will remove it from a pin, it just means you can remove it from a pin).
A counterpoint to this is the Send
trait. It doesn't mean that you will send the value, it just means you can send it.
@anp I agree that CanUnpin
isn't a great name, I'm attempting to do my damndest to figure out a better name! It seems few still feel this needs to be renamed though as all suggestions seem to get shot down for basically the same reasons I feel like Unpin
needs renaming in the first place.
Also as a reminder any stabilization rides the trains like all other changes, and with a release next week this definitely won't make it into that release and will next be a candidate for the next release. That means we have 7 weeks, nearly two months, to land all this to ensure it gets in ASAP. While I agree urgency is necessary it's just "let's not forget about this urgency", not "let's resolve this before Monday" urgency.
@stjepang I, too, have grown used to Unpin
now after thinking about it so much! All alternative names at this point seem quite lackluster, so I'm getting to the point where I would settle for better documentation. I would ideally like to find someone who doesn't know much about the Pin
apis to read said documentation afterwards, though, to double check it's sufficient for learning.
I'd also like to go ahead and formally block stabilization on improved documentation, as it feels especially critical for this issue.
@rfcbot concern improving-the-documentation
Concretely what I feel needs better documentation is:
P
's DerefMut
implementation "behaving reasonably" isn't documented on Pin::new_unchecked
.unsafe
function should have a code example of why it is unsafe, or at least a clear explanation of a sequence of steps that can go wrong.Pin
is and what it means. I think they'd benefit from information like "how this isn't general immovable types but a form of that" or "how is Pin
used in practice, for example with futures". Unpin
in either the documentation or code examples. For example it sounds like many combinators in futures have to handle this, and similarly some combinators specifically require this. Some example use cases of how one works with generics and how they play out might help understanding Unpin
quite a bit.I'm also curious if others have concrete actionable items to add to the documentation too!
Next I'd also like to formally block on the self
-question with as_ref
, but I suspect this will be quick to resolve. (again sorry for not remembering to bring this up sooner)
@rfcbot concern self-methods
This proposes as_ref
, as_mut
, get_ref
, get_mut
, into_ref
, and set
to all be methods on the Pin
type. The proposal mentions that we don't do this for smart pointers because of method clashes, but cites that experience shows today's implemented Pin
API isn't consistent and it ends up often just getting in the way.
These method names, however, are very short and sweet names and are quite common to find throughout the ecosystem. The libs team has run into a never ending series of bugs in the past where we add trait implementations and such and it causes clashes with existing trait methods in various crates. These methods seems especially high risk because of their names. Additionally, it's unclear whether we could ever add more methods to Pin
in the future as even if the names now end up working out any future name added has a high risk of collision.
I'd like to just be sure that we rethink this divergence from convention, I'm personally particularly worried about the consequences and I think it'd be great if we could find a middle-ground which would avoid these hazards.
And while I'm at it, one last thing I've noticed, and again I think this'll be pretty quick to resolve
@rfcbot concern box-pinnned-vs-box-pin
Has the name Box::pin
been considered for Box::pinned
? With the existence of Pinned
and/or PhantomPinned
it seems like it'd be neat if the name could match the return type!
@alexcrichton Has anything in particular persuaded you against MoveFromPin
as an option? I see you were favorable towards it earlier (and multiple people seemed to like it as well). For the record, the direct objections I could remember, or quickly find by Ctrl+F-ing, are that "names that are whole phrases ... would be very unidiomatic" (@withoutboats) and that it's "far too wordy" (@cramertj).
I have to admit that motivations in the vein of "I have grown used to ... now after thinking about it so much" make me rather uneasy. Humans can get used to and post-hoc-rationalize essentially anything. No matter what the name is, it would be a major surprise if we didn't eventually end up getting used to it. (This is the very reason why we choose obviously-suboptimal names like existential type
as temporaries in other cases, to try to avoid them becoming permanent.)
Accidentally privileging the perspectives of experienced users (which we all are!) over others who will be coming at things with fresher brains is the kind of mistake I think we need to be ever-vigilant against making, as hard as it is.
While it certainly isn't the style of the std lib, I also feel like CanUnpin
somehow makes it way more clear how that particular piece fits together with the other ones.
It is impossible to invent a name which describes the meaning of Unpin
to a beginner without reading documentation. Send
and Sync
are also hard to understand for a beginner, but they look nice and concise, as well as Unpin
. If you don't know what these names mean, you must read their docs.
@valff Indeed that is correct, however there are quite a few people who have read the docs and understand how pinning works, yet the Unpin
name still causes them significant mental difficulty, whereas the other names cause less.
@glaebhoerl oh sorry to clarify I'm personally more of a fan of MoveFromPin
or CanUnpin
than the current Unpin
. Just trying to help move to a conclusion!
I tried understanding the current proposal, maybe this sort of progress report can shine some additional light on naming as well.
Pin
to a stable address until Drop
puts additional requirements on the creator of a Pin
. I would have found it useful if those necessary preconditions for invoking unsafe fn new_unchecked(pointer: P) -> Pin<P>
were mentioned directly. Understanding how a Pin
can be created helps hugely in understanding their concept, imo.we agreed that we are ready to stabilize them with no major API changes.
, contains a lot of old api. At the same time, it also contains a lot of insight about Drop
and pin projection that is relevant. It also contains an explicit formulation of the additional guarantee referred to in bullet point one that is not found in this report. This mix of outdated and fresh information was a bit confusing, consider moving all of the information into this issue or indicating outdate paragraphs.slight extension
, I expected there to be some kind of way to opt out of it. Since pinning once carries a type state on that pointer until it is dropped, some way to go back from this type state to a 'normal' state. In fact, that was what I thought Unpin
to do at first but I think Unpin
is actually a bit stronger. It says the the pinned state can be freely turned into an unpinned state, at will. It's fine that the current interface is minimal in this regard but Unpin
could be confused with an operation on the type that removes some inner invariant that requires the pin state (like a soft version of Drop
).Clarification: I personally favour MoveFromPin
over the other names but it is arguably artificial and clunky.
The major actionable concrete actionable item that comes to mind, besides the documentation improvements already requested by @alexcrichton , would be explaining the reasoning behind the field rule for pin projection in map_unchecked
and map_unchecked_mut
. Something along the lines of:
A pin promises to not move the referred data until it is dropped. Since the fields of a struct are dropped after their containing struct, their pin state extends beyond the
Drop
implementation of the structs itself. Projecting to a field promises not to move from it within the structs destructor.
What about the name ElidePin
for the auto derived marker trait?
It would capture that the type can always be treated as if or behaves as if it were unpinned. And !ElidePin
seems to be quite clear as well, although that might be subjective.
The standard marker doesn't have a perfect name for understanding pinning as well. Pinned
seems to evoke the notion that the containing struct itself is pinned but pinning applies to pointers and only after an explicit action. This is not quite as critical but not insignificant either, especially since it could be the type that is first encountered in an implementation for a self-referential type.
My general opposition to MoveFromUnpin
/RemoveFromUnpin
is just that they're far too wordy and would regress the ergonomics of manual Future
implementations. CanUnpin
/DePin
both seem fine to me-- I wish it was more clear that Unpin
follows the pattern of Read
/Write
/etc. but it seems that's not intuitive to folks so I'll +1 anything that makes this clearer without looking like syntactic salt.
I agree NotWhateverUnpinBecomes
is probably the best name for Pinned
. That said, Adhere
means both to heed and to stick to. :slightly_smiling_face:
CanUnpin/DePin both seem fine to me-- I wish it was more clear that Unpin follows the pattern of Read/Write/etc
I think one of the things that makes Unpin
hard, unlike Read
is that it is a marker trait. Read
is easy to understand because there is a method Read::read
- everything you need to know is right there in the trait
. If x
is Read
I understand that I can call x.read()
- similarly write
for Write
, etc. It's harder to explain that X
implementing Unpin
means that Pin<Ptr<X>>
implements DerefMut
- which means that you can treat it as if it were just X
.
Read is easy to understand because there is a method Read::read
If only we could add always-applicable methods in auto trait
definitions + GATs, we could have Unpin::unpin
-- fn unpin<P: DerefFamily>(self: Pin<P::Deref<Self>>) -> P::Deref<Self>
). .....on second thought, I don't think that's going to make anyone less confused ;)
(on a more serious note, I'd support Pin::unpin
for going from Pin<P<T>>
to P<T>
)
I'd support Pin::unpin for going from Pin<P
> to P
This confuses two terms in my head, just like the current name itself. unpin
sounds very much like reversing the guarantees of the type state, for a comparison that would be as if there was a method fn unborrow(&'_ T)
or fn unmove(??)
. Since pin
provides guarantees until some memory representing a type is Drop::drop
ed, we don't really reverse the state, the type merely guarantees that all other representations uphold equivalent guarantees and we can thus ignore this. This is also the main difference I see between marker traits and something like io::Read
. One enables operations for the compiler or language, while the other enables operations for the programmer.
Furthermore, it is a major point that this cannot currently be accurately represented just using correct typing. To call the operation unpin
makes it sound like there was a converse function pin
. It being a function also slightly wrongly hints that such an unpin operation is somehow bound to computational work, for example into_pointer()
would be bettter in this regard by also following a more established naming convention.
Lastly, I think there's potential for types specifically having an unpin
function with computational work. A type with very special inner invariants might be able to 'fix' its inner state in such a way that it can offer an interface fn unpin(self: Pin<&'a mut T>) -> &'a mut
, where it willingly forfeits all guarantees of Pin
for some lifetime 'a
. In that case, both of the above points no longer apply. Such a function could be envisioned as if it had equivalent effect to dropping and reconstructing in the same memory location (thus actually removing the type state). And it may involve computation, e.g. by moving some self references into dynamic allocations.
It would be unfortunate if a confusing name would also make it harder for library designers and implementors to choose non-confusing names, by conflating these two ideas.
My general opposition to MoveFromUnpin/RemoveFromUnpin is just that they're far too wordy and would regress the ergonomics of manual Future implementations.
I don't think that's true, especially for MoveFromPin
, which seems reasonable to type for me - and with any kind of smart or dumb autocompletion the problem is mostly nonexistent anyway.
One important part of ergonomics should be also readability and understandability of code. Rust already earned a bit of criticism in the past for abbreviating things aggressively (fn
, mut
, etc), which makes some code harder to read. For things that for a concept that is even more complicated to grasp and which fulfills a niche purpose for most users, using a more verbose and descriptive name should be totally acceptable.
@rfcbot resolve naming-of-Unpin
Ok I've been stewing for awhile now and have also talked with @withoutboats in person about this. I've come around that I'm now personally comfortable at least with the name Unpin
as the marker trait. As a result, I'm going to remove a blocking objection (although if others on the libs team feel differently, please let us know!)
The main reason that I've come around to Unpin
is basically what @cramertj already said above, it's the most idiomatic name for this trait that we've been able to come up with. All other trait names I've seen proposed don't meet the idiomatic bar that Unpin
lives up to (or at least in my opinion).
While I still believe there are better names for "I just saw the name of this trait and I want to know what it does", I don't think that's the right problem to solve here. It's clear to me at this point that we can't, at this time at least, figure out a different name for this trait which is also idiomatic, but rather we're switching around on trait names that solve a different problem. Understanding Unpin
is just like Send
and Sync
, the documentation needs to be read to understand fully what's going on.
As another clarifying point, I listed a few "blocking objections" but they're moreso TODO items than blocking objections per se. I just don't have a great way of otherwise navigating such a long thread! In that vein I think it's fine for a stabilization PR to be posted at any time now with FCP lapsed for a week or so now. The final points about self/pin/pinned can be briefly discussed there (if even necessary, they could be left as the proposal above).
Documentation I think especially is not a prerequisite for stabilization in this case. We've got a full cycle (6 weeks) to add docs before these types hit stable and we've got even longer to shore up the documentation here before the full async/await story hits stable. That's loads of time to improve what's there now, and what's there now is already quite useful!
What does "idiomatic" even mean here? What's unidiomatic about Reloc[ate]
, Escape
, Evade
, Pluck
, or Roam
? They're all verbs, and none of them can be mistaken for having the double-negative problem.
@alexcrichton Is there a reason why Unpin
is considered more idiomatic than Depin
or DePin
?
I think Depin
is a very solid alternative, and it doesn't cause me the same mental difficulties as Unpin
.
One simple reason might be that depin isn't a word and unpin is. Personally, I don't have any trouble understanding Unpin
. I think it fits with the naming of the other marker traits.
@jimmycuadra There's a lot of names in Rust which aren't "real" words, including in the stdlib.
I would be surprised if that was considered a significant reason to choose one name over another.
@Pauan It is a significant reason. As a native English speaker, Depin
sounds to me like we forgot that the word for unpinning exists and tried to make one up. It seems glaringly wrong to me gramatically: the English word for "depinning" is "unpinning."
A good analogy would be if we had APIs that said "delock" instead of "unlock".
@jaredr by "idiomatic" I mean following the established conventions of the standard library already like the (previously mentioned) Read
and Write
traits. We have a convention of not-wordy-names, verbs, short where possible, and appropriate for the situation.
Names like you've suggested are all possible, but Unpin
(or so I feel at least) is the most appropriate for this particular action ("you can unpin this type" guarantee). The others, while loose synonyms of Unpin
, I think are largely only renaming away from the word "unpin" and sometimes don't convey the same connection with Pin
as Unpin
does.
@Pauan I agree with @withoutboats that Depin
doesn't sound like it jives as well as Unpin
.
@aturon noted in https://github.com/rust-lang/rfcs/pull/2592#issuecomment-438873636 that:
The use of
Pin
is no more of an implementation detail than the choice of&self
versus&mut self
; in the long run, it's likely thatPin
will itself be a separate mode of reference with language support. In all these cases, what's being conveyed in the signature is the permissions being granted to the callee, withPin
forbidding moving out of the reference.
Ostensibly this refers to a native &pin
type or something... but idk how this squares with Pin<&mut T>
and such. In my conversations with @withoutboats and in particular @cramertj they were not at all sure about the idea of a separate mode of reference with language support and how we could get there from Pin<T>
.
Before stabilizing pin
, it would be prudent to reconcile these views to ensure that we are on the same page wrt. this critical piece of infrastructure; so I'd mainly like for Aaron to expand on this and for boats and Taylor to then weigh in.
The others, while loose synonyms of Unpin, I think are largely only renaming away from the word "unpin" and sometimes don't convey the same connection with Pin as Unpin does.
@alexcrichton this is actually a good thing, i think? Like Send
to move/copy (=) operation, Sync
to borrow (&) operation. Maybe such connection is actually causing more confusion?
@crlf0710 it may! I'm not sure I would agree with such a connection though myself. Send
and Sync
are more about what they're enabling ("send"ing types to other threads and "sync"hronizing access across threads), and when naming them we didn't try to avoid naming them close to one operation or another
@alexcrichton exactly! so maybe this trait should also be about what it is enabling. ("moving(A verb here) out of a Pin
). I'm not native english speaker, but i still think "unpining" out of a pin
is a little...weird?
@crlf0710 But what the trait enables is not <moving>
out of a Pin, its <moving out of a Pin>
. The problem with move and synonyms for move is that they imply the trait controls the ability of the type to move at all, which it does not do. The connection to Pin is vital to understanding what the trait actually does.
So the most idiomatic name for this trait would be a verb which means "to move out of a pin," for which the word "Unpin" is the most obvious to me by a lot. Here's Wiktionary's definition of "unpin":
- To unfasten by removing a pin.
- (transitive, computing, graphical user interface) To detach (an icon, application, etc.) from the place where it was previously pinned.
@withoutboats thanks for explaining! Actually i think i can accept this Unpin
name or any name rust team finally decides on, i don't want to block the stabilization of Pin types at all, and i absolutely understands the concerns about "moving" etc.
It just feels there's a very slightest "repetition" somewhere. and i have to persuade my self: it doesn't mean "unpinnable", but the opposite. I guess after some time i'll get used to it if that's the final decision.
At the same time please allow me to make my last suggestion: actually I think the Detach
verb above mentioned is also quite nice, though a little too general. As i said, i'm not native speaker, so i can't speak for others. Please just see it as a little idea.
I was thinking about safe disjoint pinned projections, and came up with the idea that may enable it. This is a macro to simulate matching against pinned mutable lvalues. With this macro we can write
pin_proj! { let MyStruct { field1, field2 } = a_pinned_mutable_reference; }
to decompose a Pin<&mut MyStruct>
into pinned mutable references to the fields.
To make this safe and usable, we'll need two other things:
Kite
type to mark "always-Unpin
" fieldsUnpin
unsafeThe latter is needed for the safety of the projection. Without this, we can define "Kite
with pinned projections" safely, which is actually wrong.
With this in mind, I propose to make Unpin
unsafe before stabilization to leave a room for other useful operations to be safe.
@qnighy Its possible for a type to violate pinning guarantees in its Drop
implementation, which makes Drop
equivalently powerful to Unpin
. Making Unpin
unsafe does not make any additional code safe, because Drop
is also safe, and that can't be changed. We've talked about this a lot on the tracking issue, and its been mentioned on this thread as well.
Fundamentally, pin projections cannot be made safe without the ability to assert certain negative bounds that can't be expressed in Rust today.
@crlf0710
At the same time please allow me to make my last suggestion: actually I think the Detach verb above mentioned is also quite nice, though a little too general. As i said, i'm not native speaker, so i can't speak for others. Please just see it as a little idea.
Detach seems a lot better than names like Move and Relocate, but it seems to conflict with other possible uses of the detach metaphor (similar to how Escape could conflict with "escape analysis" etc). There's only so many metaphors we have about how data can relate to other data in computer science, which makes me think of another new advantage of Unpin: by cleaving tightly to the "pin" metaphor, it doesn't occupy space for future metaphoric language we may need to use for a different purpose, the way names like Detach or Escape or Relocate could.
@withoutboats I have no particular preference for any name or much investment in this bikeshed... but I'm curious; what other purpose would Detach
fit (if you speculate...)?
@Centril I don't have a clear use case in mind, but I could imagine some use case related to destructuring for example.
@withoutboats Yeah that makes sense; cheers!
@withoutboats I'm not sure if the Wiktionary entry is the best motivation to justify the naming of Unpin
to enable move from a pin
. The physical representation metaphors suffer from the fact that moving in Rust is not taking away an object in the world. To be more precise, 'upinning' a pinned pointer does not give me control over the referred to memory, it's allocation and validity as an object representation of T
are still required and guaranteed by the pointer semantics. Hence unpinning does not give a fully controlled representation of T
, as unboxing would. And, a dictionary entry defining unpinning
in terms of moving
is in fact part of the confusion more than the justification of such a name.
In the proposed API, none of the methods has such an effect (and it's not in the scope of pin either). I don't see a safe way for example to transition Pin<Box<T>>
to Box<T>
(and by extension to T
) and nor am I sure if Unpin
should have such strong possibilities. I'm not exactly sure where all the difference would be. But as far as I understood, Pin
applies to some memory location while Unpin
guarantees that nothing in T
s representation relies on pin guarantees. However, would that be the same as being able to take out of and forget the memory entirely? I think not. I'll think of a more concrete example.
Edit: More concretely, even if T
is Unpin
, can I rely on some instance of T::drop(&mut)
being called on the memory that was pinned, before that memory is deallocated? As far as I can tell from the formalism, the answer should be yes, but the name Unpin
communicates to me the opposite.
Edit 2: Rc
allows one to observe Pin<&T>
but for T: Unpin
still not have drop
called on the original memory location. Keep a reference alive outside the pin, then after the pin was dropped you can move out with Rc::try_unwrap
. https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=dec6f6c6d2c0903d87a4a9cefe50a0ca That effectively answers the question through existing mechanisms but does it work as intended?
Perhaps IgnorePin
? If T
is IgnorePin
you can treat Pin<Box<T>>
as &mut T
essentially ignoring the presence of Pin
(AIUI also ignoring the Box
). To ignore is a verb, I guess that IgnorePin
isn't but I'm not sure what it is. IgnorePin
is descriptive of what it allows, it's not descriptive of the constraints placed upon the type, but neither is Unpin
.
@wmanley I had a very similar idea, ElidePin
, in some comment above although back then I had not been able to express concretely why that one felt more precise. But I concur that 'one verb' is the style guide for Rust marker traits. Even though it also allows a more natural negation in the form of !ElidePin
/!IgnorePin
, it is not optimal.
@withoutboats Follow up question: Since pin seems specified in terms of the underlying memory, how does Pin interact with ZST? Due to Pinned
being a ZST, even a ZST may be either Unpin
or not. I would intuitively say that its memory representation is never formally invalidated, thus T::drop(&mut self)
need never be called, quite similar to how one could build a pin out of &mut 'static _
memory for example from a leaked Box
. At the same time, that may all be wrong and I see how another interpretation would be possible. I feel like these settings deserve attention in the documentation.
Is it safe to create a Pin<P<T>>
with a T: !Unpin
and then immediately unpin the value, if it's guaranteed that nothing observed the pinning? Is it only undefined behavior if the pinned value is moved after it has had a poll-like method called on it?
@HeroicKatora Unfortunately, it's possible today to implement Drop
for a type that could be !Unpin
due to generics, e.g.:
struct S<T>(T); // `!Unpin` if `T: !Unpin`
impl<T> Drop for S<T> { ... }
@cramertj Thank you, just realized that as well.
@rfcbot fcp merge Feature name:
pin
Stabilization target: 1.32.0 Tracking issue: #49150 Related RFCs: rust-lang/rfcs#2349This is a proposal to stabilize the
pin
library feature, making the "pinning" APIs for manipulating pinned memory usable on stable.(I've tried to write this proposal as a comprehensive "stabilization report.")
Stabilized feature or APIs
[std|core]::pin::Pin
This stabilizes the
Pin
type in thepin
submodule ofstd
/core
.Pin
is a fundamental, transparent wrapper around a generic typeP
, which is intended to be a pointer type (for example,Pin<&mut T>
andPin<Box<T>>
are both valid, intended constructs). ThePin
wrapper modifies the pointer to "pin" the memory it refers to in place, preventing the user from moving objects out of that memory.The usual way to use the
Pin
type is to construct a pinned variant of some kind of owning pointer (Box
,Rc
, etc). The std library owning pointers all provide apinned
constructor which returns this. Then, to manipulate the value inside, all of these pointers provide a way to degrade towardPin<&T>
andPin<&mut T>
. Pinned pointers can deref, giving you back&T
, but cannot safely mutably deref: this is only possible using the unsafeget_mut
function.As a result, anyone mutating data through a pin will be required to uphold the invariant that they never move out of that data. This allows other code to safely assume that the data is never moved, allowing it to contain (for example) self references.
The
Pin
type will have these stabilized APIs:impl<P> Pin<P> where P: Deref, P::Target: Unpin
fn new(pointer: P) -> Pin<P>
impl<P> Pin<P> where P: Deref
unsafe fn new_unchecked(pointer: P) -> Pin<P>
fn as_ref(&self) -> Pin<&P::Target>
impl<P> Pin<P> where P: DerefMut
fn as_mut(&mut self) -> Pin<&mut P::Target>
fn set(&mut self, P::Target);
impl<'a, T: ?Sized> Pin<&'a T>
unsafe fn map_unchecked<U, F: FnOnce(&T) -> &U>(self, f: F) -> Pin<&'a U>
fn get_ref(self) -> &'a T
impl<'a, T: ?Sized> Pin<&'a mut T>
fn into_ref(self) -> Pin<&'a T>
unsafe fn get_unchecked_mut(self) -> &'a mut T
unsafe fn map_unchecked_mut<U, F: FnOnce(&mut T) -> &mut U>(self, f: F) -> Pin<&'a mut U>
impl<'a, T: ?Sized> Pin<&'a mut T> where T: Unpin
fn get_mut(self) -> &'a mut T
Trait impls
Most of the trait impls on
Pin
are fairly rote, these two are important to its operation:impl<P: Deref> Deref for Pin<P> { type Target = P::Target }
impl<P: DerefMut> DerefMut for Pin<P> where P::Target: Unpin { }
std::marker::Unpin
Unpin is a safe auto trait which opts out of the guarantees of pinning. If the target of a pinned pointer implements
Unpin
, it is safe to mutably dereference to it.Unpin
types do not have any guarantees that they will not be moved out of aPin
.This makes it as ergonomic to deal with a pinned reference to something that does not contain self-references as it would be to deal with a non-pinned reference. The guarantees of
Pin
only matter for special case types like self-referential structures: those types do not implementUnpin
, so they have the restrictions of thePin
type.Notable implementations of
Unpin
in std:impl<'a, T: ?Sized> Unpin for &'a T
impl<'a, T: ?Sized> Unpin for &'a mut T
impl<T: ?Sized> Unpin for Box<T>
impl<T: ?Sized> Unpin for Rc<T>
impl<T: ?Sized> Unpin for Arc<T>
These codify the notion that pinnedness is not transitive across pointers. That is, a
Pin<&T>
only pins the actual memory block represented byT
in a place. Users have occassionally been confused by this and expected that a type likePin<&mut Box<T>>
pins the data ofT
in place, but it only pins the memory the pinned reference actually refers to: in this case, theBox
's representation, which a pointer into the heap.std::marker::Pinned
The
Pinned
type is a ZST which does not implementUnpin
; it allows you to supress the auto-implementation ofUnpin
on stable, where!Unpin
impls would not be stable yet.Smart pointer constructors
Constructors are added to the std smart pointers to create pinned references:
Box::pinned(data: T) -> Pin<Box<T>>
Rc::pinned(data: T) -> Pin<Rc<T>>
Arc::pinned(data: T) -> Pin<Arc<T>>
Notes on pinning & safety
Over the last 9 months the pinning APIs have gone through several iterations as we have investigated their expressive power and also the soundness of their guarantees. I would now say confidently that the pinning APIs stabilized here are sound and close enough to the local maxima in ergonomics and expressiveness; that is, ready for stabilization.
One of the trickier issues of pinning is determining when it is safe to perform a pin projection: that is, to go from a
Pin<P<Target = Foo>>
to aPin<P<Target = Bar>>
, whereBar
is a field ofFoo
. Fortunately, we have been able to codify a set of rules which can help users determine if such a projection is safe:(Foo: Unpin) implies (Bar: Unpin)
: that is, if it is never the case thatFoo
(the containing type) isUnpin
whileBar
(the projected type) is notUnpin
.Bar
is never moved during the destruction ofFoo
, meaning that eitherFoo
has no destructor, or the destructor is carefully checked to make sure that it never moves out of the field being projected to.Foo
(the containing type) is notrepr(packed)
, because this causes fields to be moved around to realign them.Additionally, the std APIs provide no safe way to pin objects to the stack. This is because there is no way to implement that safely using a function API. However, users can unsafely pin things to the stack by guaranteeing that they never move the object again after creating the pinned reference.
The
pin-utils
crate on crates.io contains macros to assist with both stack pinning and pin projection. The stack pinning macro safely pins objects to the stack using a trick involving shadowing, whereas a macro for projection exists which is unsafe, but avoids you having to write the projection boilerplate in which you could possibly introduce other incorrect unsafe code.Implementation changes prior to stabilization
Unpin
from the prelude, removepin::Unpin
re-exportAs a general rule, we don't re-export things from multiple places in std unless one is a supermodule of the real definition (e.g. shortening
std::collections::hash_map::HashMap
tostd::collections::HashMap
). For this reason, the re-export ofstd::marker::Unpin
fromstd::pin::Unpin
is out of place.At the same time, other important marker traits like Send and Sync are included in the prelude. So instead of re-exporting
Unpin
from thepin
module, by putting in the prelude we make it unnecessary to importstd::marker::Unpin
, the same reason it was put intopin
.Currently, a lot of the associated function of
Pin
do not use method syntax. In theory, this is to avoid conflicting with derefable inner methods. However, this rule has not been applied consistently, and in our experience has mostly just made things more inconvenient. Pinned pointers only implement immutable deref, not mutable deref or deref by value, limiting the ability to deref anyway. Moreover, many of these names are fairly unique (e.g.map_unchecked
) and unlikely to conflict.Instead, we prefer to give the
Pin
methods their due precedence; users who need to access an interior method always can using UFCS, just as they would be required to to access the Pin methods if we did not use method syntax.get_mut_unchecked
toget_unchecked_mut
The current ordering is inconsistent with other uses in the standard library.
impl<P> Unpin for Pin<P>
This impl is not justified by our standard justification for unpin impls: there is no pointer direction between
Pin<P>
andP
. Its usefulness is covered by the impls for pointers themselves.This futures impl will need to change to add a
P: Unpin
bound.Pin
asrepr(transparent)
Pin should be a transparent wrapper around the pointer inside of it, with the same representation.
Connected features and larger milestones
The pin APIs are important to safely manipulating sections of memory which can be guaranteed not to be moved out. If the objects in that memory do not implement
Unpin
, their address will never change. This is necessary for creating self-referential generators and asynchronous functions. As a result, thePin
type appears in the standard libraryfuture
APIs and will soon appear in the APIs for generators as well (#55704).Stabilizing the
Pin
type and its APIs is a necessary precursor to stabilizing theFuture
APIs, which is itself a necessary precursor to stabilizing theasync/await
syntax and moving the entirefutures 0.3
async IO ecosystem onto stable Rust.cc @cramertj @RalfJung