Reconstructor looking for method "reconstruct" in an IObjectConstructor(ClassDictConstructor), which only specifies method "construct"

[donaldjosephsmith]

My apologies if this is the wrong place to report this; I've just started experimenting with this and it seems like this is a bug but it might be my own user-error. I'm having trouble unpickling (protocol 0, though I also have problems with 2) a fairly simple custom Python class into Java. To get right to the point, load_reduce pops an IObjectConstructor from the pickle stack, which ends up being a Reconstructor instance, and then calls construct on it. In the end, however, this particular construct is looking for a reconstruct method on the reduce argument, which doesn't exist in ClassDictConstructor. In fact, the only class anywhere that implements a reconstruct seems to be TimeZoneConstructor. This of course results in a NoSuchMethod exception. I think I may be missing something... but I don't know what. It seems like ClassDictConstructor is supposed to have a reconstruct method but it doesn't. I was under the impression from the documentation that instances of custom Python classes would simply be unpickled into a Map of the data members.

Here's the pickle text up to the REDUCE where it dies:

(dp0
I0
ccopy_reg
_reconstructor
p1
(cgmGroup2
TreeNodeData
p2
c__builtin__
object
p3
Ntp4
Rp5

Here's the Python class in question:

class TreeNodeData (object):
    def __init__ (self, xpos, *children):           
        self.children = children                    
        self.parent = None                          
        self.size = 1                               
        self.position = (xpos, None)                
        return                                      

And, the debug view after I tracked down the problem:

[irmen]

What Python code did you use to come up with that protocol 0 pickle? What Python version did you use? Can't you just use protocol 2 or higher?

(also note that the copy_reg reconstructor thingy has been added in https://github.com/irmen/Pyrolite/pull/22 just to enable pickling of datetime.tzinfo objects. The "reconstruct" method is actually part of another class: TimeZoneConstructor.)

[donaldjosephsmith]

Python version is 2.7.6 (on Ubuntu 14.04)

Just using the following simple method:

def exportGroupData(self, filename):
  file = open(filename, 'wb')
  pickle.dump(self.treeData, file, 0)

where treeData is a dict mapping either an integer or a string to a TreeNodeData object.

[donaldjosephsmith]

If it helps, I've also reproduced by pickling an instance of a very simple "test" class as well:

class test (object):
  def __init__(self, something):
    self.something = something

t = test('asdf')
f = open('test.pickle', 'w')
pickle.dump(t, f, 0)

Here's the Java side:

public class PickleTest {
    public static void main(String[] args) {
        Unpickler unpickler = new Unpickler();
        Object data;
        try {
            data = unpickler.load(new FileInputStream("/Volumes/dsmith/test.pickle"));
            System.out.println(data.toString());
        }
        catch (PickleException | IOException e) {
            e.printStackTrace();
        }
    }
}

[irmen]

Simple solution: stop using pickle protocol 0, use pickle protocol 2 or higher. Then your objects will be deserialized in the java side as a hashtable (actually a ClassDict which extends HashMap) , as you expect.

Another solution: write your own class reconstructor to teach Pyrolite how to deserialize your copy_reg pickled custom class. As already explained the only class it now supports for this nasty pickle construct is datetime.tzinfo. To unpickle your own custom classes you have to tell it how to do that by adding your own custom (re)constructor using Unpickler.registerConstructor().

[irmen]

By the way, for reference the following unit tests may be interesting: https://github.com/irmen/Pyrolite/blob/master/java/src/test/java/net/razorvine/pickle/test/UnpicklerComplexTest.java#L189 https://github.com/irmen/Pyrolite/blob/master/java/src/test/java/net/razorvine/pickle/test/UnpicklerComplexTest.java#L210

[donaldjosephsmith]

Thanks!