search

rust-lang / unsafe-code-guidelines

Forum for discussion about what unsafe code can and can't do
https://rust-lang.github.io/unsafe-code-guidelines
Apache License 2.0
658 stars 57 forks source link

Semver guarantees for data layouts #242

Open the8472 opened 4 years ago

the8472 commented 4 years ago

Spun off from https://github.com/rust-lang/unsafe-code-guidelines/issues/36#issuecomment-664609200 and previous.

Currently layout guarantees either come from explicit choices (adding repr) or follow from other guarantees (e.g. all-pub struct). #36 aims to introduce a new compiler guarantee that automatically applies to some repr(Rust) structs and would make field reordering a breaking change for unsafe code relying on that compiler guarantee.

The question is whether unsafe code could reasonably expect these guarantees to be upheld by other crates across minor version bumps.

If yes this would place a new requirement on crate authors and those requirements can be subtle if a struct contains generic fields which would make it sometimes homogeneous and sometimes not.

If no then crate authors would have to explicitly provide this guarantee either via comment or some annotation that can be checked by the compiler or semver linters and unsafe code may need compile-time checks that its assumptions still remain true. There also is the risk that unsafe code authors may conflate compiler guarantees with semver guarantees.

RalfJung commented 4 years ago

So the general question here is: when semver guarantees that old clients of the library keep working, to what extend does that apply to clients using unsafe? My personal expectation is "not at all", i.e., semver only covers things that are observable by safe code.

This question lives on the intersection of UCG and... whatever group is responsible for ironing out semver. Wasn't there some library API design group?

RalfJung commented 4 years ago

Cc @rust-lang/libs this is a UCG question that overlaps with semver / API design questions. Your input would be appreciated.

dtolnay commented 4 years ago

Is the concern that someone takes a library type with private fields and transmutes to/from [T; N] to access the fields?

They should not do that. If the fields are private you don't get to assume what they are.

RalfJung commented 4 years ago

Is the concern that someone takes a library type with private fields and transmutes to/from [T; N] to access the fields?

I think that was @the8472' concern, yes.

They should not do that. If the fields are private you don't get to assume what they are.

Agreed. Is there some place this is written down or so? It seems to be worth documenting, given that the question was raised.

the8472 commented 4 years ago

No, not private fields. Even a repr(Rust) type with all-pub fields currently does not guarantee its ordering thus it wouldn't be safe for unsafe code to make assumptions about field ordering. #36 would change that, the question is whether that also means unsafe code authors could automatically rely on that to be covered by semver guarantees, i.e. whether crate authors have to start preserving the order because someone else might use it in unsafe code.

Mark-Simulacrum commented 4 years ago

It's probably worth noting that the safe transmute work is currently planning on providing traits which allow libraries to indicate how much can be assumed by consumers, see https://github.com/rust-lang/project-safe-transmute/pull/5 for the latest RFC draft.

the8472 commented 4 years ago

The RFC provides a safe interface for transmutation, which is great but does not necessarily answer what authors of unsafe code can assume.

So the general question here is: when semver guarantees that old clients of the library keep working, to what extend does that apply to clients using unsafe? My personal expectation is "not at all", i.e., semver only covers things that are observable by safe code.

For an all-pub struct field offsets are observable from safe code, that doesn't guarantee that those offsets are stable.

RalfJung commented 4 years ago

For an all-pub struct field offsets are observable from safe code, that doesn't guarantee that those offsets are stable.

Fair. Just like type sizes are typically considered excluded from semver (as far as I know) even though they are observable by safe code, I would think the same applies to field offsets.