birkenfeld / serde-pickle

Rust (de)serialization for the Python pickle format.
Apache License 2.0
185 stars 27 forks source link

Recommended way to yield the parsed values? #7

Closed L0g4n closed 4 years ago

L0g4n commented 4 years ago

Hi,

this might be a little dump question -- I am still rather new to Rust -- but what is the recommended approach to yield the parsed values, i.e. get access to the underlying Rust datatypes.

In my case: After parsing I now have Value, to be more precise a Dict(BTreeMap<HashableValue::String, List(Vec<Value>) as an enum variant. This of course represents a dictionary which has a string as a key and an entry that is a map that includes a tuple.

So my question what is an idiomatic way to get the access to the underlying pure Rust datatypes? My first approach would be to pattern match to get access to the values but this seems cumbersome for nested datatypes like in my case. Or is the only way?

birkenfeld commented 4 years ago

First off, you probably know this already, but the Value type is only useful in the case where the document you're deserializing has an unknown structure. If the structure is known, it's preferable to use your own structs and derive Deserialize for them.

That said, you're right that I haven't given the Value any convenience APIs to use on top of pattern matching. I assume you'd like something akin to serde_json Value?

L0g4n commented 4 years ago

Now that you mention it - I probably should implement Deserialize for my struct since the structure is known.

L0g4n commented 4 years ago

That said, you're right that I haven't given the Value any convenience APIs to use on top of pattern matching. I assume you'd like something akin to serde_json Value?

I just looked into them and just I think that would be nice - for the "unknown structure usecase" this would increase the usability quite a bit.

birkenfeld commented 4 years ago

BTW, if your data would be valid JSON (e.g only string keys in dictionaries) you can even deserialize into serde_json::Value and use its API :)

L0g4n commented 4 years ago

Well, it is quite similar to JSON since it's a Python dict with string keys that contains lists but I do not know if parsing the decoded pickle stuff would not lead to syntax errors in JSON.

Edit: In my case not feasible since my lists contain tuples which are not valid JSON.

birkenfeld commented 4 years ago

Tuples will just deserialize to Arrays there.

L0g4n commented 4 years ago

Good to know, but I will try the "derive Deserialize approach" first. I guess that means I have to implement the trait from serde.

birkenfeld commented 4 years ago

Yes, usually it's nothing more than #[derive(Deserialize)] for each struct that appears in the (possibly nested) data structure. See https://serde.rs/derive.html

L0g4n commented 4 years ago

So, I tried the "derive Deserialize" approach just by myself, and it works pretty good 👍 After deserialization I now get debug output like this

Deserialized: RpaIndices({"all_assets.rpyc": [RpaTuple(1111638641, 1111684220, "")]})

The serde API seems pretty developer friendly, thanks for your help!

birkenfeld commented 4 years ago

Nice! Yes, serde is one of the go-to examples of good API design in Rust.