espressomd / espresso

The ESPResSo package
https://espressomd.org
GNU General Public License v3.0
225 stars 183 forks source link

Serialisation of proxy objects can be improved #4187

Open KaiSzuttor opened 3 years ago

KaiSzuttor commented 3 years ago
import io
import pickle

import espressomd
import espressomd.accumulators
import espressomd.observables

class ObsEncapsulator:
    pass

obs = espressomd.observables.ParticlePositions(ids=[0,])
acc = espressomd.accumulators.MeanVarianceCalculator(obs=obs)
enc = ObsEncapsulator()
enc.obs = obs

buff = io.BytesIO()
out = pickle.Pickler(buff)
out.dump(obs)
out.dump(acc)
out.dump(enc)

buff.seek(0)
read = pickle.Unpickler(buff)
obs = read.load()
acc = read.load()
enc = read.load()
buff.close()

print(id(obs))
print(id(acc.get_params()['obs'])) # this is not a reference to 'obs' anymore
print(id(enc.obs)) # this is still a reference to 'obs'
fweik commented 3 years ago

While this is obviously not great, this is known (and documented) behavior. We decided to accept this because there is no straight forward way to fix this. For python objects this works because pickle actively tracks the relationship between objects, for the so object this is not visible on the python level. I think there are some potentially some deeper design flaws (by me) at work here that make this particularly hard to fix, ideas welcome.

KaiSzuttor commented 3 years ago

the underlying issue is that the referenced object is not stored but the python representation gets created upon request