jamesmunns / postcard

A no_std + serde compatible message library for Rust
Apache License 2.0
930 stars 89 forks source link

Feature Request: Deserialization support for bson::Document #102

Closed TheDan64 closed 1 year ago

TheDan64 commented 1 year ago

Postcard seems to serialize https://docs.rs/bson/2.6.1/bson/struct.Document.html okay (no error at least) but when I deserialize it back, I get WontImplement. It's not really clear to me why that is, a Document is basically just a HashMap under that hood and it implements the serde traits

TheDan64 commented 1 year ago

Example:

let bytes = postcard::to_allocvec(&doc! {"abc": "def"})?;
dbg!(&bytes);
let r: Document = postcard::from_bytes(&bytes)?;
[src/main.rs:91] &bytes = [
    1,
    3,
    97,
    98,
    99,
    3,
    100,
    101,
    102,
]
Error: This is a feature that PostCard will never implement
TheDan64 commented 1 year ago

A HashMap<String, String> otoh produces identical bytes but is ok:

let bytes = postcard::to_allocvec(&{
    let mut h = HashMap::new();
    h.insert("abc".to_string(), "def".to_string());
    h
})?;
dbg!(&bytes);
let r: HashMap<String, String> = postcard::from_bytes(&bytes)?;
[src/main.rs:96] &bytes = [
    1,
    3,
    97,
    98,
    99,
    3,
    100,
    101,
    102,
]
jamesmunns commented 1 year ago

I'd have to look at the code, but I think the only thing that we return that error is for the deserialize_any option, which is usually used for unstructured things like JSON.

basically, we can't really take something like:

{
    "abc": "def"
}

and turn it into

struct {
    abc: String,
}

I'm not familiar with BSON, are fields optional/nullable in that format? It's possible that they use deserialize_any in a non-derived deserializer implementation. You may be able to work around this with serde's deserialize_with attribute to first deserialize as a HashMap, then convert that into a Document.

TheDan64 commented 1 year ago

Yeah, it's basically binary serialized JSON stored as a hashmap internally

jamesmunns commented 1 year ago

Yup, unfortunately, I don't think I can support this. You can likely add a step to deserialize it as a hashmap then convert that to a Document, but we also don't support something like serde_json::Value.

Since the Document type uses serde's deserialize_any API rather than deserializing from a sequence of key:value pairs, postcard has no way of knowing that this is the intent.

My best suggestion is to make a wrapper function for deserialization that does the conversion, and use serde's deserialize_with attributes to call out to this function.

I'll close this, but feel free to reopen if you think there's something I missed!