search

seung-lab / znn-release

Multi-core CPU implementation of deep learning for 2D and 3D sliding window convolutional networks (ConvNets).
GNU General Public License v3.0
94 stars 33 forks source link

KeyError: 'Unable to open object (Component not found)' #98

Open petteriTeikari opened 8 years ago

petteriTeikari commented 8 years ago

Hi,

I encountered the following error when trying to open my previously trained models, is there something happened somwhere inside or outsize the znn-release that the models are not compatible anymore?

stdpre:  /
Traceback (most recent call last):
  File "train.py", line 263, in <module>
    main( args )
  File "train.py", line 69, in main
    net, lc = znetio.create_net(pars)
  File "/home/petteri/testVesselNN/vesselNN/znn-release/python/front_end/znetio.py", line 454, in create_net
    lc = zstatistics.CLearnCurve( fnet )
  File "/home/petteri/testVesselNN/vesselNN/znn-release/python/zstatistics.py", line 28, in __init__
    self._read_curve( fname )
  File "/home/petteri/testVesselNN/vesselNN/znn-release/python/zstatistics.py", line 49, in _read_curve
    self.tt_it  = list( f[self.stdpre + 'test/it'].value )
  File "_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip_build_root/h5py/h5py/_objects.c:2511)
  File "_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip_build_root/h5py/h5py/_objects.c:2464)
  File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/group.py", line 166, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
  File "_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip_build_root/h5py/h5py/_objects.c:2511)
  File "_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip_build_root/h5py/h5py/_objects.c:2464)
  File "h5o.pyx", line 190, in h5py.h5o.open (/tmp/pip_build_root/h5py/h5py/h5o.c:3556)
KeyError: 'Unable to open object (Component not found)'

or with Anaconda2

stdpre:  /
Traceback (most recent call last):
  File "train.py", line 263, in <module>
    main( args )
  File "train.py", line 69, in main
    net, lc = znetio.create_net(pars)
  File "/home/petteri/testVesselNN/vesselNN/znn-release/python/front_end/znetio.py", line 454, in create_net
    lc = zstatistics.CLearnCurve( fnet )
  File "/home/petteri/testVesselNN/vesselNN/znn-release/python/zstatistics.py", line 28, in __init__
    self._read_curve( fname )
  File "/home/petteri/testVesselNN/vesselNN/znn-release/python/zstatistics.py", line 49, in _read_curve
    self.tt_it  = list( f[self.stdpre + 'test/it'].value )
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2579)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2538)
  File "/home/petteri/anaconda2/lib/python2.7/site-packages/h5py/_hl/group.py", line 164, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2579)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2538)
  File "h5py/h5o.pyx", line 190, in h5py.h5o.open (/home/ilan/minonda/conda-bld/work/h5py/h5o.c:3551)
KeyError: 'Unable to open object (Component not found)'
xiuliren commented 8 years ago

we have a standard to format all the data, including the network file. but I have modified the zstatistics to be backward compatible, should work out-of-box. Following the standard or not was set by is_stdio in your training configuration file. if you open the network file with hdfview, you should see /processing in a standard network file.

according to the stdpre: /, your network file is not a standard network. Not sure why zstatistics is not compatible with this. Could you share your network with me to reproduce?

petteriTeikari commented 8 years ago

Hi @jingpengwu the trained networks can be found from there: https://github.com/petteriTeikari/vesselNN_dataset/tree/master/experiments/VD2D_tanh

and as well from there: https://github.com/petteriTeikari/vesselNN_dataset/tree/master/experiments/VD2D3D_tanh

petteriTeikari commented 8 years ago

And yes @jingpengwu , there is no /processing in the network file

net_statistics net_current

xiuliren commented 8 years ago

@petteriTeikari You should use the net_statistics_current.h5 file rather than the net_current.h5. We have merged these two files now.

in the master branch, python zstatistics.py net_statistics_current.h5 Here is what I get:

screenshot from 2016-07-22 11 40 34

petteriTeikari commented 8 years ago

Thanks @jingpengwu for this clarification, but I actually had the problem of using the pre-trained VD2D part to start the VD2D3D from scratch

petteriTeikari commented 8 years ago

And @jingpengwu , where did the update happen as I can now for sure start training from scratch but I get the same open object -problem when trying to resume the training so the current master still incorrectly saves the network during training, or the subsequent zstatistics.py call?

xiuliren commented 8 years ago

@petteriTeikari it is this merging: https://github.com/seung-lab/znn-release/commit/1d436a84c3d68d737b60beaa880cd6da599f73ca#diff-1e9e8194b95ef941cdb5dec8cea4bc90R68