Unsafe fields

[pnkfelix]

Multiple developers have registered a desire to allow public access to fields that expose abstraction-breaking impl details (rather than following the current convention of making all such fields private to the structure in question, thus containing the abstraction-breakage to the structure's module).

See the following postponed (or otherwise closed) RFCs:

• 80

• 182

• 358

[daniel-vainsencher]

I am guessing that @ticki is relying on "mutable xor shared" and @golddranks is talking about a module that breaks it internally e.g., to provide an efficient concurrent data structure.

I think that my assumption that only writing is unsafe needs discarding. Unsafe fields should be unsafe to access, period. And I think that fields not subject to "mutable xor shared" should never be public, even if any unsafe fields are allowed to be (which I don't know about).

[ticki]

If unsafe fields are unsafe to read, they don't have the same power. Everyone would prefer to write the respective getter function instead of the field directly, sacrificing performance.

[daniel-vainsencher]

How important is the performance benefit of shaving off an accessor in debug builds?

I think the main argument for private unsafe fields is preserving memory safety at a small maintenance expense. In this case, accessing those fields through a small set of functions that ensure written and read values are indeed valid is probably a best practice anyway.

[ticki]

How important is the performance benefit of shaving off an accessor in debug builds?

Well, considering how many getters there are, I think that some gain will be there.

[ticki]

But the argument for local guarantees for invariants is much more compelling.

[Kimundi]

My thoughts on this:

• unsafe fields are not needed since you can emulate them with abstraction around the struct.

• They are a useful convenience. since they would allow expressing with one keyword what you'd need up to 4 unsafe accessor functions for:

  unsafe fn get_ref(&self) -> &T;
  unsafe fn get_mut(&mut self) -> &mut T;
  unsafe fn get_move(self) -> T;
  unsafe fn set(t: T);

• The abstraction would be useful both outside the abstraction for reducing the need for said accessors, and inside for giving a more strict self-check of the implementation.
• They should have the semantic "any access is unsafe" for the same reasons static mut has - you can't know in what state the field is.
  • This semantic would have a hurdle with types that have destructors though: How do unsafe fields get dropped, seeing how drop is a safe operation?
  • Possible solution: Unsafe fields may only contain Copy data. In combination with a copyable ManualDrop type, this would still allow all types.

[daniel-vainsencher]

@Kimundi About safety of dropping: To my understanding your concern is that any implementation of the Drop trait that accesses the unsafe fields has to include unsafe section? why is that a problem exactly? it seems to exactly encode the fact that these fields carry invariants, and any finalization done on them requires care.

Or are you concerned about memory safety when Drop is not implemented at all? IIUC, Rust does not guarantees that drops will occur, so we should not depend on them for memory safety purposes anyway. Right?

[ticki]

But it is strongly wanted (and often assumed, though not for memory safety) for destructors to be called. Not doing so would be quite counterintuitive. If destructors are not to be called in unsafe fields, I would require a !Drop.

[daniel-vainsencher]

@ticki, @Kimundi, you seem to be referring to dropping the value in the field that is marked unsafe, right? I though Kimundi was referring to a drop of the struct containing the field.

But unsafe code already should ensure (beforehand) that it is safe to drop any sensitive fields before they get dropped, right?

So the unsafe fields proposal merely helps formalize this obligation: any field that might be unsafe to drop at any time should be marked an unsafe field, and brought to a safe state before it gets dropped.

Hence, I think the compiler should drop values in unsafe fields exactly when it would in regular ones. Am I missing something? This is also an important property if we want porting modules (with unsafe to use unsafe fields) to be an easy enhancement of documentation with no associated danger.

[Kimundi]

@daniel-vainsencher: I'm referring to this case:

struct Foo {
    unsafe bar: SomeType
}

where SomeType has a destructor.

If Foo where a regular struct, then dropping an instance of Foo would call the destructor of SomeType with a &mut to the bar field.

If, however, bar is marked as a unsafe field, and thus unsafe to access, then it would be unsafe for a drop of Foo to also drop the bar field.

So either unsafe fields need to be defined as needing manual dropping, requiring to implement Drop explicitly; or they need to be defined as being Copy to ensure they don't need to be dropped at all.

Third option would be to silently not drop then, but that seems like a footgun.

[daniel-vainsencher]

@Kimundi Thank you for taking the time to write your view very explicitly. The term unsafe is being overloaded (maybe more specialized terms exist and I am not aware of them):

1) Some actions are dangerous bugs, causing UB etc. We call those unsafe or memory unsafe. 2) unsafe sections allow us to do some things in category (1), but have another desired use: The rust community seems to want to be able to trace all cases of (1) to cases of (2). 3) For this purpose, we use for example the annotation of functions as unsafe to propagate the danger signal; marking a function unsafe does not add potential for bugs!

You seem to treat accessing unsafe fields as unsafe in sense (1): actually can cause UB, like static mut variables. This could happen, for example if we allowed the compiler more liberty in optimizing around such variables. In this case, allowing the regular drop behavior (so when we drop Foo, we call drop on SomeType as well, which can be a bug, therefore the compiler should not do it. Do I understand correctly?

As far as I am concerned, unsafe fields are exactly like unsafe functions, a tool for (3). Marking a field unsafe is sufficient warning to any maintainer of the code that when the compiler does its usual thing around drops, heavy objects had better be put away. This is the same responsibility that writers of unsafe code face now, just made a bit more precise: with correctly annotated unsafe fields, the drop of other fields should never cause unsafety (in sense 1). I imagine that when it is non-trivial, this responsibility will often be discharged using an unsafe section inside the destructor of Foo. Does this make sense?

Therefore in my opinion unsafe fields should be dropped exactly when regular fields are; they should not require manual dropping, implementation of Drop, a Copy bound. All of those are tools an implementor might use to avoid unsafety in sense (1), but its his choice.

[Kimundi]

@daniel-vainsencher

I think you convinced me :) To put what you said in different words, would you agree that unsafe fields could be defined as this:

• Marking a field of a struct as unsafe requires an unsafe block for any direct access of the field.
  • This also includes construction of the struct.
• The implementation of the struct still needs to ensure that the field is always in a state where destroying the struct does not cause memory unsafety.
  • Specifically, if the field contains a type with a destructor, it needs to be in a safe-to-call-drop state after the drop of the containing struct.
  • This would be a invariant that needs to be kept in mind at initialization/modification time of the field value

[daniel-vainsencher]

That is a concise and precise definition that I agree with. On May 7, 2016 8:38 AM, "Marvin Löbel" notifications@github.com wrote:

@daniel-vainsencher https://github.com/daniel-vainsencher

I think you convinced me :) To put what you said in different words, would you agree that unsafe fields could be defined as this:

• Marking a field of a struct as unsafe requires an unsafe block for any direct access of the field.
  • This also includes construction of the struct.
• The implementation of the struct still needs to ensure that the field is always in a state where destroying the struct does not cause memory unsafety.
  • Specifically, if the field contains a type with a destructor, it needs to be in a safe-to-call-drop state after the drop of the containing struct.
  • This would be a invariant that needs to be kept in mind at initialization/modification time of the field value

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/rust-lang/rfcs/issues/381#issuecomment-217632817

[petrochenkov]

I think unions will need to eventually support very similar capability, but with the opposite sign - making fields safe. Unsafe blocks is a huge burden especially on something that is statically known (to the user, but not to the language) to be safe. A good example is unions like LARGE_INTEGER (again). cc https://github.com/rust-lang/rust/issues/32836

[RalfJung]

I feel like https://github.com/rust-lang/rust/issues/41622 is another point where unsafe fields could have helped: If we make it so that types with unsafe fields to NOT get any automatic instances for unsafe OIBITS, that would have prevented accidentally implementing Sync for MutexGuard<Cell<i32>>.

Really, there is no way to predict how the invariant of a type with unsafe fields could interact with the assumptions provided by unsafe traits, so implementing them automatically is always dangerous. The author of a type with unsafe fields has to convince himself that the guarantees of an unsafe traits a satisfied. Not getting these instances automatically is an ergonomics hit, but the alternative is to err on the side of "what could possibly go wrong", which is not very Rust-y.

[nikomatsakis]

@RalfJung good point. I am pretty strongly in favor of unsafe fields at this point. The only thing that holds me back is some desire to think a bit more about the "unsafe" model more generally.

[Kixunil]

Bit OT: I wonder how hard would it be to implement a way to express invariants maintained by said struct and insert lints (and possibly runtime checks in debug mode) about them. Probably it wouldn't help with broken Send and Sync but it might be interesting e.g. in case of Vec (len <= capacity).

[sighoya]

I've a question, I want to be able to point to the same entity with one pointer being owned, the other being mutably borrowed. My example is to implement a single linked list which is optimized for appending elements to the end, so I have a first_node and a last_node which can potentially be overlapping:

pub struct SingleLinkedList<T>
{
    unsafe first_node: OptionNode<T>,
    unsafe last_node: &OptionNode<T>,
    pub length: mut u32,
}

Would that be possible? Or is there any workaround to this?

[RustyYato]

@sighoya This won't be possible, as it would break Rust's aliasing model

The only thing this does is makes accessing these fields unsafe, nothing more.

[RalfJung]

This would (finally) be resolved by https://github.com/rust-lang/rfcs/pull/3458.