Does K3D-jupyter support joblib.dump()?

[tfunatomi]

Description

I'm a big fan of this great module. I appreciate your efforts.

I tried to use joblib.dump() to store the current object, but it failed.

I know that snapshot can store as a static HTML, but is there any way to store the object that can be loadable from another program?

What I Did

I used joblib.dump() as follows:

import joblib
import k3d
from k3d import platonic

plot = k3d.plot()
dodec_1 = platonic.Dodecahedron()
dodec_2 = platonic.Dodecahedron(origin=[5, -2, 3], size=0.5)
plot += dodec_1.mesh
plot += dodec_2.mesh

joblib.dump(plot, './dump.joblib')
# joblib.dump(plot.objects, './dump.joblib') also failed.

It crashed as follows:

TypeError                                 Traceback (most recent call last)
Cell In [1], line 11
      8 plot += dodec_1.mesh
      9 plot += dodec_2.mesh
---> 11 joblib.dump(plot, './dump.joblib')
     12 joblib.dump(plot.objects, './dump.joblib')

File /usr/local/lib/python3.10/dist-packages/joblib/numpy_pickle.py:553, in dump(value, filename, compress, protocol, cache_size)
    551 elif is_filename:
    552     with open(filename, 'wb') as f:
--> 553         NumpyPickler(f, protocol=protocol).dump(value)
    554 else:
    555     NumpyPickler(filename, protocol=protocol).dump(value)

File /usr/lib/python3.10/pickle.py:487, in _Pickler.dump(self, obj)
    485 if self.proto >= 4:
    486     self.framer.start_framing()
--> 487 self.save(obj)
    488 self.write(STOP)
    489 self.framer.end_framing()

... omitted ...

File /usr/local/lib/python3.10/dist-packages/joblib/numpy_pickle.py:355, in NumpyPickler.save(self, obj)
    352     wrapper.write_array(obj, self)
    353     return
--> 355 return Pickler.save(self, obj)

File /usr/lib/python3.10/pickle.py:578, in _Pickler.save(self, obj, save_persistent_id)
    576 reduce = getattr(obj, "__reduce_ex__", None)
    577 if reduce is not None:
--> 578     rv = reduce(self.proto)
    579 else:
    580     reduce = getattr(obj, "__reduce__", None)

TypeError: cannot pickle '_hashlib.HMAC' object

[artur-trzesiok]

Hi!

I'm not familiar with joblib but for storing data we have example: https://github.com/K3D-tools/K3D-jupyter/blob/main/examples/snapshots.ipynb

plot = get_plot()
plot.display()
data = plot.get_binary_snapshot()

with open('binary_snapshot.k3d', 'wb') as f:
    f.write(data)

and loading:

plot2 = k3d.plot()
with open('binary_snapshot.k3d', 'rb') as f:
    plot2.load_binary_snapshot(f.read())
plot2.display()

That use internal binary format of k3d - without any overhead

[tfunatomi]

@artur-trzesiok Thank you for your advice!

So, it might be easy to support joblib.dump() by just defining Plot.__getstate__() and Plot.__setstate__(state) to call get_binary_snapshot() and load_binary_snapshot(data) . https://docs.python.org/3/library/pickle.html#pickle-inst

[tfunatomi]

On my environment, just adding the following lines in k3d.Plot seems working.

    def __getstate__(self):            
        return self.get_binary_snapshot()

    def __setstate__(self,data):
        self.__init__()
        self.load_binary_snapshot(data)

[tfunatomi]

413 : Ready for this

[artur-trzesiok]

Great finding and solution!

[artur-trzesiok]

Hi! Please check a 2.15.3 version of k3d. It is already available :)