search

lemon24 / reader

A Python feed reader library.
https://reader.readthedocs.io
BSD 3-Clause "New" or "Revised" License
438 stars 36 forks source link

pyserde for canonical serialization #247

Closed adsharma closed 3 years ago

adsharma commented 3 years ago

More context: https://github.com/yukinarit/pyserde/issues/143

Have you considered pyserde + msgpack?

lemon24 commented 3 years ago

Hi, thank you for opening this!

I have not, mainly due to the "no dependencies" requirement.

I looked at the linked issue (and the msgpack one) and the pyserde docs, and I think it would satisfy all other requirements. Aside from what's mentioned in the issues, dict order should be always alphabetical; this can likely be done with a custom class serializer.

Given _hash_utils.py is already written and pretty short, and I still have the "no dependencies" requirement, it's unlikely I would actually switch.

Part of why I didn't look for a full serialization solution was that I knew I'd never need to deserialize the objects; as far as I can tell, that's where the problem starts to have a lot of nuance, and I'd prefer to use a library instead of doing my own thing.

Is there a specific reason for suggesting pyserde or why the current approach doesn't work for you, or is this more of an exploratory kind of thing?

An aside: I am vaguely familiar with Rust's Serde, but not enough to be able to tell if pyserde translates its philosophy into Python. That said, the API looks pretty nice, and I like the approach of generating a (de)serializer once, instead of always doing introspection (seems to be shared by mashumaro, which I also didn't know about).

If I ever need (de)serialization, it's likely I'll use one of these; thanks for bringing them to my attention.

adsharma commented 3 years ago

No specific reason. Your blog showed up when I searched for "python dataclass serialization stable hash". Hopefully the next person searching for it can discover this additional tidbit.

The reason why I was looking into this is to add immutable hashes to https://github.com/adsharma/raft and try out some of the recent ideas from Heidi Howard about immutable storage for raft.