birkenfeld / serde-pickle

Rust (de)serialization for the Python pickle format.
Apache License 2.0
188 stars 28 forks source link

Reset memoize to support streaming multiple objects. #18

Closed jeroenbakker-atmind closed 3 years ago

jeroenbakker-atmind commented 3 years ago

By default (value_)from_reader closes the input stream. It is possible to deserialize multiple pickle objects from a single stream by implementing a custom reader.

This patch will reset the memoize at the beginning when a new object is deserialized. Any refs would then point to the correct object inside the memoize.

We checked several approaches but this approach seems to fit best (support most protocols) however this might not be the obvious place.

birkenfeld commented 3 years ago

Thanks, I have no objections to this version. The doctests for 1.41.1 seem to fail because a Cursor of an array isn't Read - could you make the input a slice in the example?

jeroenbakker-atmind commented 3 years ago

Yes, we will check on 1.41.1

birkenfeld commented 3 years ago

Thanks!