lava-nc / lava-dl

Deep Learning library for Lava
https://lava-nc.org
BSD 3-Clause "New" or "Revised" License
155 stars 69 forks source link

Compiled netx hdf5 models cannot be serialized. #253

Open Gavinator98 opened 1 year ago

Gavinator98 commented 1 year ago

Describe the bug netx hdf5 models create an error when attempting to save a compiled executable

To reproduce current behavior Steps to reproduce the behavior:

  1. Load and compile a netx hdf5 model
    
    from lava.lib.dl import netx
    from lava.magma.core.run_configs import Loihi2HwCfg
    from lava.utils.serialization import save

net = netx.hdf5.Network(net_config='some_model.net') executable = net.compile(run_cfg=Loihi2HwCfg())

save(processes=[net], executable=executable, filename="compiled_model.pkl")

2. Produces this error

h5py objects cannot be pickled


As a workaround, the h5py object can be removed before saving.
```python
net.net_config = None 

Expected behavior netx.hdf5.Network objects work with the save() function without modification.

Environment (please complete the following information):

PhilippPlank commented 7 months ago

Thank you for reporting this issue. We are aware that this is not possible and it is an h5py issue in general. We are testing some mitigation strategies, but h5py is explicitly preventing to use pickle and changing the h5py package is not a general solution. If you need that, you can do it though (remove the exception in the h5py code).