birkenfeld / serde-pickle

Rust (de)serialization for the Python pickle format.
Apache License 2.0
188 stars 28 forks source link

Unsupported builtin type list. #15

Closed jeroenbakker-atmind closed 3 years ago

jeroenbakker-atmind commented 3 years ago

Hi,

In our usecase we pickle ctypes to simple lists using pickle reduce.

def reduce_euler(ob):
    return (list, ([ob[0], ob[1], ob[2]],))

This results in writing a builtin type to the pickle stream. This isn't supported by serde-pickle.

See here the diff that fixes this issue for us.

diff --git a/src/de.rs b/src/de.rs
index 1e60169..0957606 100644
--- a/src/de.rs
+++ b/src/de.rs
@@ -37,6 +37,7 @@ enum Global {
     Set,         // builtins/__builtin__.set
     Frozenset,   // builtins/__builtin__.frozenset
     Bytearray,   // builtins/__builtin__.bytearray
+    List,        // builtins/__builtin__.list
     Encode,      // _codecs.encode
     Other,       // anything else (may be a classobj that is later discarded)
 }
@@ -789,6 +790,8 @@ impl<R: Read> Deserializer<R> {
                 Value::Global(Global::Set),
             (b"__builtin__", b"frozenset") | (b"builtins", b"frozenset") =>
                 Value::Global(Global::Frozenset),
+            (b"__builtin__", b"list") | (b"builtins", b"list") =>
+                Value::Global(Global::List),
             (b"__builtin__", b"bytearray") | (b"builtins", b"bytearray") =>
                 Value::Global(Global::Bytearray),
             _ => Value::Global(Global::Other),
@@ -835,6 +838,15 @@ impl<R: Read> Deserializer<R> {
                     _ => self.error(ErrorCode::InvalidValue("bytearray() arg".into())),
                 }
             }
+            Value::Global(Global::List) => {
+                match self.resolve(argtuple.pop()) {
+                    Some(Value::List(items)) => {
+                        self.stack.push(Value::List(items));
+                        Ok(())
+                    }
+                    _ => self.error(ErrorCode::InvalidValue("set() arg".into())),
+                }
+            }
             Value::Global(Global::Encode) => {
                 // Byte object encoded as _codecs.encode(x, 'latin1')
                 match self.resolve(argtuple.pop()) {  // Encoding, always latin1
birkenfeld commented 3 years ago

Fixed by #16.