Pickling References

[andrewkatson]

This is more just a question about functionality. I would just like to better understand so that I can consider writing something myself. I checked the backlog and did not find something explicitly about this.

Essentially I would like the following to work.

import dill
import weakref

class Example: 

   def __init__(self, val):
        self.val = val

a_val = 10
persistent = Example(a_val)

temp_ref = weakref.proxy(persistent)

pickled_ref = dill.dumps(temp_ref)

persistent.val = 20

restored_ref = dill.loads(pickled_ref)

assert persistent.val == restored_ref.val

I understand why the weakref should die if the persistent object no longer exists but why does it have to die in this case?

[mmckerns]

Because, when you serialize the weakref without the thing it points to, it breaks the reference... so it should fail. If you want it to persist, you need to also serialize the persistent object.

>>> temp_ref = weakref.proxy(persistent)
>>> pickled_ref = dill.dumps((persistent, temp_ref))
>>> restored_obj, restored_ref = dill.loads(pickled_ref)
>>> restored_obj.val
20
>>> restored_ref.val
20
>>> 

[andrewkatson]

Yeah I am just asking if the object persists anyways is there a way to modify the implementation

[mmckerns]

If it would be possible... I'd think very fragile. The thing is, logically, a weak reference needs something to refer to... so I'm not sure what use case there is for having it work otherwise. Do you have something in mind that needs different behavior?

[andrewkatson]

Yes I have one. So I am using protocol buffers and I want to pass around references as part of an event driven system. I am just trying to keep a strict API where you can only react to events. In this system, I need to pass a reference of Actor A to Actor B but Actor B obviously cannot own them. Since my events are just protocol buffers a string serialization of a reference is how I would do that. In this case my guarantees of use are they are on the same thread/fiber, Actor A will exist as long as I need that weakref serialized, and both Actors are in the same binary. It is a pretty strict and admittedly artificial use case but I am doing this to get deterministic replay.