google / flatbuffers

FlatBuffers: Memory Efficient Serialization Library
https://flatbuffers.dev/
Apache License 2.0
22.66k stars 3.2k forks source link

Rust Push::alignment Incorrect for Structs #8150

Open tustvold opened 9 months ago

tustvold commented 9 months ago

Consider a struct of the form

struct FieldNode {
  length: long;
  null_count: long;
}

The following code is generated for Push

impl<'b> flatbuffers::Push for FieldNode {
    type Output = FieldNode;
    #[inline]
    unsafe fn push(&self, dst: &mut [u8], _written_len: usize) {
        let src =
            ::core::slice::from_raw_parts(self as *const FieldNode as *const u8, Self::size());
        dst.copy_from_slice(src);
    }
}

This therefore uses the default impl of Push::alignment which is

fn alignment() -> PushAlignment {
    PushAlignment::new(align_of::<Self::Output>())
}

Unfortunately the definition of FieldNode is

pub struct FieldNode(pub [u8; 16]);

Which has an alignment of 1.

The net result is that the writer does not provide the correct alignment guarantees for structs, which causes the verifiers of some implementations to fail - https://github.com/apache/arrow-rs/issues/5052.

tustvold commented 8 months ago

@CasperN perhaps you have some thoughts on this?

evgenyx00 commented 8 months ago

@tustvold btw, meanwhile based on your findings, I did a "hack" in flatbuffer create_vector/create_vector_from_iter and changed the alignment to 8.

jpochyla commented 7 months ago

So the codegen in the official Rust backend is generating code that produces wrong data (probably coming from misunderstanding of Rust alignment rules) and none of the maintainers care?

github-actions[bot] commented 1 month ago

This issue is stale because it has been open 6 months with no activity. Please comment or label not-stale, or this will be closed in 14 days.

evgenyx00 commented 1 month ago

This is still an issue, adding a comment to avoid closure of this problem.