offset_of! for unsized structs

[est31]

This doesn't work with memoffset:

struct Foo {
    bar: f32,
    baz: [f32],
}
offset_of!(Foo, bar);

Even though this (unsound) code works:

struct Foo {
    bar: f32,
    baz: [f32],
}
let foo_ptr: &Foo = unsafe { mem::zeroed() };
let foo_offs = (&foo_ptr.bar) as *const _ as usize;

I ran into this problem when trying to port glium to memoffset in https://github.com/glium/glium/pull/1782. The gpgpu example has relied on offset of calculations supporting unsized structs. I could work around the issue by changing the gpgpu example, but that isn't really perfect.

[RalfJung]

Hm, good question. Even with https://github.com/rust-lang/rfcs/pull/2582 I am not sure how to do this UB-free.

@Amanieu @rkruppe @eddyb any ideas?

[Amanieu]

In the general case it isn't possible to determine the offset of a field in an unsized struct. For example:

trait Foo {}
struct Bar {
    a: i32,
    b: dyn Foo,
}

The offset of b is going to be different depending on the alignment of the underlying type that implements Foo. In fact rustc will use the vtable of b to find its alignment and determine the offset of b to access the field.

[pema99]

I'm trying to do some GPGPU stuff with glium, and running into this issue. The use of memoffset means that implement_uniform_block! doesn't work for unsized structs. Is there a simple workaround, or will I have to fork?

[RalfJung]

I don't see how a fork can help. I don't think there is a way to compute the offset of a field in an unsized struct. It is certainly not possible for the last (unsized) field since there the question is ill-posed (it's like asking for the size of an unsized struct -- the unsized field does not have a fixed offset). For the other fields it is possible in theory but I am not sure how to implement it, in particular how to ensure that it's not the last field that is being asked for.

Maybe one day finally someone will actually go through the effort of adding offset_of as a compiler-understood primitive...

[pema99]

I don't see how a fork can help. I don't think there is a way to compute the offset of a field in an unsized struct. It is certainly not possible for the last (unsized) field since there the question is ill-posed (it's like asking for the size of an unsized struct -- the unsized field does not have a fixed offset). For the other fields it is possible in theory but I am not sure how to implement it, in particular how to ensure that it's not the last field that is being asked for.

Maybe one day finally someone will actually go through the effort of adding offset_of as a compiler-understood primitive...

I've just reverted to using the unsound code posted by OP. I don't care about the unsoundness for my use case.

[RalfJung]

Uh... no that's not a solution, that's asking the compiler to break your code please. :(

Lucky enough, the compiler detects this and aborts the program with a panic.

[est31]

I won't switch glium to an UB having alternative, but I would really like this issue to be resolved. As for forking, I don't care. It's @pema99 's own project and there are use cases where UB is less of an issue than in others.

A native offset_of functionality, however it is exposed, sounds really good.

[RalfJung]

FWIW, if you must, then please use raw pointers rather than references. That's at least less UB...

A native offset_of functionality, however it is exposed, sounds really good.

Indeed! I hope someday someone who actually needs it will be motivated enough to add it. I know at least one lang team member is in support.

[avl]

I don't see how a fork can help. I don't think there is a way to compute the offset of a field in an unsized struct. It is certainly not possible for the last (unsized) field since there the question is ill-posed (it's like asking for the size of an unsized struct -- the unsized field does not have a fixed offset). ...

I'm curious, how can this be? I understand you can't ask about the 'end' of the last field, since it can't be known until runtime. But I'm surprised that the field doesn't always start att the same location. My mental model is that the last field would start at the first memory location allowed by alignment requirements, and would then have a runtime-variable length. Is this not how it is?

Specifically, for this struct:

struct Example {
    x: u8,
    y: [u64]
}

Wouldn't 'y' always start at offset 8? Or is the case that if y is empty, the whole struct is size 1, not 8, and y is without start entirely?

[RalfJung]

The issue arises with dyn Trait tails, whose alignment is not statically known -- and the alignment influences the offset.