[Bug]: `Semantic(None, "invalid type: bytes, expected bytes")`

[RCasatta]

Is there an existing issue for this?

• [X] I have searched the existing issues

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Current Behaviour

I am getting the error Semantic(None, "invalid type: bytes, expected bytes")

Expected Behaviour

deserialization works

Environment Information

Linux 6.5.4-76060504-generic #202309191142~1695998943~22.04~070916d SMP PREEMPT_DYNAMIC Fri S x86_64 x86_64 x86_64 GNU/Linux

Steps To Reproduce

The error is launched from here https://github.com/enarx/ciborium/blob/2ac91ce8a398bcdbaa250a92831786ffe42052a2/ciborium/src/de/mod.rs#L379 because the header is bytes but the len of scratch is not enough

~~I can reproduce the error in a downstream crate with a type implementing custom serde Serialize/Deserialize: https://github.com/ElementsProject/rust-elements/pull/179/commits/45bd2bf274a7ed44da1d8fa65e80749d2f188db7 This type is longer than 4k bytes~~

When I try to reproduce without the downstream crate using something like Wrap(vec) where vec is bigger than 4k bytes, things work because in this line:

https://github.com/enarx/ciborium/blob/2ac91ce8a398bcdbaa250a92831786ffe42052a2/ciborium/src/de/mod.rs#L369

the scratch space has grown to accommodate the len of the bytes, (it is 8k even though it is initialized to 4k) while in the case of the error it's probably not growing for some reason I could not grasp yet.

[RCasatta]

Following code could reproduce:

#[cfg(test)]
mod tests {
    use std::fmt;

    use serde::{de, Deserialize, Serialize};

    #[derive(PartialEq, Eq, Debug)]
    struct Wrap(Vec<u8>);

    impl Wrap {
        fn from_slice(slice: &[u8]) -> Result<Wrap, String> {
            Ok(Wrap(slice.to_vec()))
        }
        fn serialize(&self) -> Vec<u8> {
            self.0.clone()
        }
    }

    impl Serialize for Wrap {
        fn serialize<S: ::serde::Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
            s.serialize_bytes(&self.serialize())
        }
    }

    impl<'de> Deserialize<'de> for Wrap {
        fn deserialize<D: ::serde::Deserializer<'de>>(d: D) -> Result<Wrap, D::Error> {
            d.deserialize_bytes(BytesVisitor::new("a bytestring", Wrap::from_slice))
        }
    }

    pub struct BytesVisitor<F> {
        expectation: &'static str,
        parse_fn: F,
    }

    impl<F, T, Err> BytesVisitor<F>
    where
        F: FnOnce(&[u8]) -> Result<T, Err>,
        Err: fmt::Display,
    {
        pub fn new(expectation: &'static str, parse_fn: F) -> Self {
            BytesVisitor {
                expectation,
                parse_fn,
            }
        }
    }

    impl<'de, F, T, Err> de::Visitor<'de> for BytesVisitor<F>
    where
        F: FnOnce(&[u8]) -> Result<T, Err>,
        Err: fmt::Display,
    {
        type Value = T;

        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
            formatter.write_str(self.expectation)
        }

        fn visit_bytes<E: de::Error>(self, v: &[u8]) -> Result<Self::Value, E> {
            (self.parse_fn)(v).map_err(E::custom)
        }
    }

    #[test]
    fn ciborium() {
        let a = Wrap(vec![0u8; 5000]);
        let bytes = cib_vec(&a).unwrap();
        let b: Wrap = ciborium::from_reader(&bytes[..]).unwrap();
        assert_eq!(a, b);
    }

    fn cib_vec<S: Serialize>(s: &S) -> Result<Vec<u8>, ciborium::ser::Error<std::io::Error>> {
        let mut vec = vec![];
        ciborium::ser::into_writer(s, &mut vec)?;
        Ok(vec)
    }
}