sharksforarms / deku

Declarative binary reading and writing: bit-level, symmetric, serialization/deserialization
Apache License 2.0
1.14k stars 55 forks source link

Seemingly nonsensical vector count/read_size error #402

Closed sempervictus closed 10 months ago

sempervictus commented 10 months ago

Am running into something strange trying to make protocol parsing more idiomatic:

#[derive(Debug, PartialEq, DekuRead, DekuWrite)]
#[deku(endian = "big")]
struct Header {
    pub(super) pkt_length: u16,
    pub(super) pkt_checksum: u16,
    pub(super) pkt_type: u8,
    pub(super) flags: u8,
    pub(super) checksum: u16
}
#[derive(Debug, PartialEq, DekuRead, DekuWrite)]
struct Packet {
    pub(super) header: Header,
    #[deku(count = "header.pkt_length")]
    pub(super) body: Vec<u8>
}

The header parses just fine in testing, the body however, not so much - Packet::try_from(bytes.as_ref()).unwrap(); and from_bytes all raise some form of called `Result::unwrap()` on an `Err` value: Incomplete(NeedSize { bits: 8 }) despite count being defined in the macro. I have tried using bytes_read and bytes per the attribute documentation, but not having any luck there either. The header parses just fine: Header { pkt_length: 157, pkt_checksum: 0, pkt_type: 1, flags: 0, checksum: 0 } so i know that header.pkt_length is 157 and have no idea why that value is not being used. Figuring it might be a nested reference problem, i flattened the struct to:

struct Packet {
    #[deku(endian = "big")]
    pub(super) pkt_length: u16,
    #[deku(endian = "big")]
    pub(super) pkt_checksum: u16,
    #[deku(endian = "big")]
    pub(super) pkt_type: u8,
    #[deku(endian = "big")]
    pub(super) flags: u8,
    #[deku(endian = "big")]
    pub(super) checksum: u16,
    #[deku(count = "pkt_length")]
    pub(super) body: Vec<u8>
}

all to the same effect (seems i can't set endianness at the struct level when it has a vec in it). Clearly i'm doing something wrong here though its not clear to me exactly what from comparing this to how the docs structure the Vec<u8> read length constraint using count.

sempervictus commented 10 months ago

Closing as this actually is a bug on my end - cargo clean and rustup update seemed to do the trick.

sharksforarms commented 10 months ago

Glad you found a solution! (I'm not sure why a cargo clean & rustup update would solve the issue though :( )

Next time, it would help to provide an example w/ assert on what you're expecting, that'd make it easier for us to reproduce, such as the following:

use deku::prelude::*;

#[derive(Debug, PartialEq, DekuRead, DekuWrite)]
#[deku(endian = "big")]
struct Header {
    pkt_length: u16,
    pkt_checksum: u16,
    pkt_type: u8,
    flags: u8,
    checksum: u16,
}
#[derive(Debug, PartialEq, DekuRead, DekuWrite)]
struct Packet {
    header: Header,
    #[deku(count = "header.pkt_length")]
    body: Vec<u8>,
}

fn main() {
    env_logger::init();

    let input = &[0x00, 0x02, 0xDE, 0xAD, 0xAB, 0xCD, 0xBE, 0xEF, 0xFF, 0xFF];
    let ((rest, _bit_offset), pkt) = Packet::from_bytes((input, 0)).unwrap();
    assert!(rest.is_empty());
    assert_eq!(
        pkt,
        Packet {
            header: Header {
                pkt_length: 0x0002,
                pkt_checksum: 0xDEAD,
                pkt_type: 0xAB,
                flags: 0xCD,
                checksum: 0xBEEF,
            },
            body: vec![0xFF, 0xFF],
        }
    );
}

Another useful way to debug is using the logging feature, that way you can see trace output from deku

$ RUST_LOG=trace cargo run --example test --features logging
[2024-01-14T02:43:57Z TRACE test] Reading: Packet.header
[2024-01-14T02:43:57Z TRACE test] Reading: Header.pkt_length
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: requesting 2 bytes
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: returning [00, 02]
[2024-01-14T02:43:57Z TRACE test] Reading: Header.pkt_checksum
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: requesting 2 bytes
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: returning [de, ad]
[2024-01-14T02:43:57Z TRACE test] Reading: Header.pkt_type
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: requesting 1 bytes
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: returning [ab]
[2024-01-14T02:43:57Z TRACE test] Reading: Header.flags
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: requesting 1 bytes
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: returning [cd]
[2024-01-14T02:43:57Z TRACE test] Reading: Header.checksum
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: requesting 2 bytes
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: returning [be, ef]
[2024-01-14T02:43:57Z TRACE test] Reading: Packet.body
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: requesting 1 bytes
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: returning [ff]
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: requesting 1 bytes
[2024-01-14T02:43:57Z TRACE deku::reader] read_bytes: returning [ff]