jam1garner / binrw

A Rust crate for helping parse and rebuild binary data using ✨macro magic✨.
https://binrw.rs
MIT License
620 stars 36 forks source link

Zero copy or zero alloc optimization? #288

Closed ileixe closed 2 months ago

ileixe commented 2 months ago

Hi,

We've invested the feasibility to replace our hand-write binary interpretation, and want to ask some of optimization technique.

  #[bitfield]
  #[binrw]
  #[br(map = Self::from_bytes)]
  #[derive(Default)]
  pub struct ProgramHeader {
      version: B4,
      protocol: B4,
      flags: u8,
      // Number of Op.
      count: u16,
  }

  #[repr(C)]
  #[binrw]
  #[brw(little)]
  #[derive(Default)]
  pub struct Program {
      header: ProgramHeader,
      #[br(count = header.count())]
      ops: Vec<Op>,
  }

This is our protocol layout and our concerns are

  1. Zero alloc

Regarding this part,

 #[br(count = header.count())]
      ops: Vec<Op>

We've used Vector here and it makes me worried about decoding performance. Before, we just reads byte stream and uses Op inplace without collecting them via Vector.

I guess there might be no way to have variadic length of data without allocation, but want to ask owner's thought.

  1. (Strict) Zero copy

Also, it's similar to previous concern. Can we have any chance to reduce copy? I know strict zero copy is only available via in-place transmute which requires somehow different binary encoding technique (e.g. rkyv), but still want to ask as well. We are affording some unsafety to some extent (We're embedded system anyway)

Thanks

(Lastly, I want to mention that even there's no such way, I would like to give weight for this crate because of much better usability, so please interpret this issue to seek little possibility.)