Can't serialize HashMaps with struct keys

[kazimuth]

Example: https://play.rust-lang.org/?gist=dc61dd0b07e76a9f602500f36e8aa35d&version=stable

Panics:

     Running `target/debug/playground`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value:
ErrorImpl { code: KeyMustBeAString, line: 0, column: 0 }', /checkout/src/libcore/result.rs:906:4

I was able to workaround this with serialize_with and deserialize_with attributes on the relevant fields of my struct:

use serde::{Serialize, Deserialize, Serializer, Deserializer};

type StructyMap = HashMap<ThingA, ThingB>;

fn serialize_structymap<S: Serializer> (map: &StructyMap, s: S) -> Result<S::Ok, S::Error> {
    map.iter().map(|(a,b)| (a.clone(),b.clone())).collect::<Vec<(_,_)>>().serialize(s)
}
fn deserialize_structymap<'de, D: Deserializer<'de>>(d: D) -> Result<StructyMap, D::Error> {
    let vec = <Vec<(ThingA, ThingB)>>::deserialize(d)?;
    let mut map = StructyMap::default();
    for (k, v) in vec {
        map.insert(k, v);
    }
    Ok(map)
}

struct ContainsMap {
    #[serde(serialize_with = "serialize_structymap")]
    #[serde(deserialize_with = "deserialize_structymap")]
    map: StructyMap
}

But that's a pretty fragile / copy-intensive solution, and ideally wouldn't be necessary.

[dtolnay]

Yeah JSON only allows string keys in a map and I don't think there is a safe default for how to handle them when we see them other than failing to serialize. Some people expect the key to be encoded as a nested JSON string like {"{\"r\":1}":283091}, some people expect the map to transform into an array of pairs [[{"r":1},283091]] like in your code.

If you are concerned about performance of the copy-intensive solution, here is a more verbose but efficiently streaming implementation with no copies. You could take the module here and publish it as a crate sort of like serde-tuple-vec-map.

#[macro_use]
extern crate serde_derive;

extern crate serde;
extern crate serde_json;

use std::collections::HashMap as Map;

#[derive(Deserialize, Serialize, Hash, PartialEq, Eq, Debug)]
struct Color {
    r: u8,
    g: u8,
    b: u8
}

#[derive(Serialize, Deserialize, Default, Debug)]
struct Kazimuth {
    #[serde(with = "map_as_pairs")]
    map: Map<Color, usize>,
}

mod map_as_pairs {
    use std::fmt;
    use std::marker::PhantomData;
    use serde::ser::{Serialize, Serializer};
    use serde::de::{Deserialize, Deserializer, Visitor, SeqAccess};

    pub fn serialize<K, V, M, S>(map: M, serializer: S) -> Result<S::Ok, S::Error>
    where
        K: Serialize,
        V: Serialize,
        M: IntoIterator<Item = (K, V)>,
        S: Serializer,
    {
        serializer.collect_seq(map)
    }

    pub fn deserialize<'de, K, V, M, D>(deserializer: D) -> Result<M, D::Error>
    where
        K: Deserialize<'de>,
        V: Deserialize<'de>,
        M: Default + Extend<(K, V)>,
        D: Deserializer<'de>,
    {
        struct MapVisitor<K, V, M> {
            keys: PhantomData<K>,
            values: PhantomData<V>,
            map: PhantomData<M>,
        }

        impl<'de, K, V, M> Visitor<'de> for MapVisitor<K, V, M>
        where
            K: Deserialize<'de>,
            V: Deserialize<'de>,
            M: Default + Extend<(K, V)>,
        {
            type Value = M;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("a sequence of key-value pairs")
            }

            fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
            where
                A: SeqAccess<'de>,
            {
                let mut map = M::default();
                while let Some((k, v)) = seq.next_element()? {
                    map.extend(Some((k, v)));
                }
                Ok(map)
            }
        }

        deserializer.deserialize_seq(MapVisitor {
            keys: PhantomData,
            values: PhantomData,
            map: PhantomData,
        })
    }
}

fn main() {
    let mut example = Kazimuth::default();
    example.map.insert(Color {r: 1, g: 2, b: 3}, 283091usize);
    example.map.insert(Color {r: 1, g: 2, b: 100}, 333usize);

    let s = serde_json::to_string(&example).unwrap();
    println!("{}", s);

    let example2: Kazimuth = serde_json::from_str(&s).unwrap();
    println!("{:#?}", example2);
}

[EmreAtes]

This example does not cause any error when writing to a file, but it just stops writing and returns:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2015&gist=fb623a637e1e7c3ab81ec09ff06ccb23

[realcr]

@dtolnay : I was wondering, should it be possible to detect this problem during compile time instead of runtime? I bumped into this problem a few times as well, and this was a behaviour I wasn't expecting from a Rusty crate.

My question is actually divided into two:

1. Is there anything a user of serde can do to have a more reliable compile time guarantees with respect to this issue?
2. Is it a goal for serde to become panic free during serialization / deserialization?

[tzcnt]

For anyone finding this issue in 2022: I wrote the crate serde_json_any_key that IMHO is a great solution for this.

It can do what serde-tuple-vec-map does, with the #[serde(with = "any_key_vec")] attribute.

It differs from vectorize in that vectorize creates a JSON array, whereas serde_json_any_key creates a JSON map where the key is a stringified version of the key type.

Serde-tuple-vec-map only supports Rust vec types, and vectorize only supports Rust map types; my crate supports both and can de/serialize them interchangeably. My crate also exposes more functionality in that it can serialize a standalone map or vec structure without requiring a new type to annotate with #[serde(with)]. It also does all serialization in one pass without creating an intermediate vec (vectorize uses 2 passes).