serde-rs / serde

Serialization framework for Rust
https://serde.rs/
Apache License 2.0
8.81k stars 747 forks source link

Flattened fields with shared inner fields don't deserialize properly #2719

Open CryZe opened 3 months ago

CryZe commented 3 months ago

If I have an outer struct that has two optional inner structs that are flattened, then the second one doesn't deserialize properly if it shares any fields with the struct of the field before it.

use serde_derive::Deserialize;

fn main() {
    let json = r#"{
        "shared": "foo",
        "unique_b": "bar"
    }"#;
    let b: B = serde_json::from_str(json).unwrap();
    let outer: Outer = serde_json::from_str(json).unwrap();
    dbg!(b);
    dbg!(outer);
}

#[derive(Debug, Deserialize)]
struct Outer {
    #[serde(flatten)]
    a: Option<A>,
    #[serde(flatten)]
    b: Option<B>,
}

#[derive(Debug, Deserialize)]
struct A {
    shared: String,
    unique_a: String,
}

#[derive(Debug, Deserialize)]
struct B {
    shared: String,
    unique_b: String,
}

So in the example they both share the shared field. The problem comes from the code that's generated for the Outer type's visit_map implementation:

let mut __collect = _serde::__private::Vec::<
    _serde::__private::Option<(
        _serde::__private::de::Content,
        _serde::__private::de::Content,
    )>,
>::new();
while let _serde::__private::Some(__key) =
    _serde::de::MapAccess::next_key::<__Field>(&mut __map)?
{
    match __key {
        __Field::__other(__name) => {
            __collect.push(_serde::__private::Some((
                __name,
                _serde::de::MapAccess::next_value(&mut __map)?,
            )));
        }
    }
}
let __field0: Option<A> = _serde::de::Deserialize::deserialize(
    _serde::__private::de::FlatMapDeserializer(
        &mut __collect,
        _serde::__private::PhantomData,
    ),
)?;
let __field1: Option<B> = _serde::de::Deserialize::deserialize(
    _serde::__private::de::FlatMapDeserializer(
        &mut __collect,
        _serde::__private::PhantomData,
    ),
)?;
_serde::__private::Ok(Outer {
    a: __field0,
    b: __field1,
})

The problem is that there's an intermediate __collect buffer that partially gets consumed by the attempt to deserialize A (in particular the shared field). This makes it impossible for B to deserialize properly.

The content being taken out of the buffer happens at the end here:

https://github.com/serde-rs/serde/blob/3bfab6ef7fc80ad73eb598687c836609c14f6f8b/serde/src/private/de.rs#L2830-L2847