search

birkenfeld / serde-pickle

Rust (de)serialization for the Python pickle format.
Apache License 2.0
185 stars 27 forks source link

TypeError: unhashable type: 'dict' upon loading serde-pickle's outputs in Python #19

Closed Calandiel closed 4 weeks ago

Calandiel commented 2 years ago

I believe it happens when structs are used as keys in hash maps and hash sets. They seem to be turned into dictionaries of their own (which is fair enough) but having them as keys in Python doesn't really work. Perhaps one could serialize them to frozensets instead? Or issue a warning if the user tries serializing something that won't deserialize in Python.

birkenfeld commented 2 years ago

Hm, good point. This is hard - I'm not sure if serializing structs differently only when they are dict keys / set items is feasible or even wanted...

Calandiel commented 2 years ago

Certainly depends on the intended use case of the library. If it's outputs are supposed to be loadable in Python, the way it works now is quite a significant limitation. If having two different serialization patterns isn't possible, perhaps serializing structs to frozensets would be a good compromise? But that gives up mutability on Python side for structs that arent keys, so it wouldn't be ideal either. Perhaps a different option would be to have two serializers, one which targets dictionaries and one which targets frozen sets?

Just throwing ideas, I'm not sure myself what would be the best way to handle it from end users perspective.

birkenfeld commented 2 years ago

It's no problem to add a different serializing mode, since 1.0 we have the SerOptions struct that can be extended with new features.

However, it's not clear to me how the alternate representation would look with named fields. Can you make a suggestion?

max-96 commented 1 year ago

Named tuples are hashable in Python, they might be a good candidate instead of dictionaries.