search

TS317 / CSC-FAQ

1 stars 4 forks source link

error when import h5py, how to fix it? #5

Open jingang-cv opened 6 years ago

jingang-cv commented 6 years ago 
import h5py

File "/appl/opt/python/2.7.10-gcc493-shared/lib/python2.7/site-packages/h5py-2.7.0-py2.7-linux-x86_64.egg/h5py/init.py", line 34, in from ._conv import register_converters as _register_converters File "h5py/h5r.pxd", line 21, in init h5py._conv (/homeappl/home/jppirhon/appl_taito/h5py-2.7.0/h5py/_conv.c:7678) File "h5py/_objects.pxd", line 12, in init h5py.h5r (/homeappl/home/jppirhon/appl_taito/h5py-2.7.0/h5py/h5r.c:3262) File "h5py/_objects.pyx", line 1, in init h5py._objects (/homeappl/home/jppirhon/appl_taito/h5py-2.7.0/h5py/_objects.c:7652) ImportError: /appl/opt/python/2.7.10-gcc493-shared/lib/python2.7/site-packages/h5py-2.7.0-py2.7-linux-x86_64.egg/h5py/defs.so: undefined symbol: H5Pget_dxpl_mpio

jingang-cv commented 6 years ago

What is strange, if we first import h5py , just a warning occurred as following. Then we can import caffe. If we import caffe first, the above error will appear.wh /appl/opt/python/2.7.10-gcc493-shared/lib/python2.7/site-packages/h5py-2.7.0-py2.7-linux-x86_64.egg/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters

jingang-cv commented 6 years ago

It seems that hdf5 and h5py are not matched because of the version of parallel or not (mpio). But how to fix it.

HenglinShi commented 6 years ago

可能你编译caffe的时候和hd5联编了，所以先load caffe后load h5会出问题，其实先load h5后load caffe可以凑合用。 想解决的话，可以试一下删除caffe下Makefile.config里/appl/opt/hdf5-serial/gcc-4.9.3/1.8.15/include和/appl/opt/hdf5-serial/gcc-4.9.3/1.8.15/lib两行后重新编译caffe，如果hdf5是必须项的话，可以自己安装hdf5或者尝试试用一下并行版的hdf5: 自己安装的话参见 https://github.com/yosinski/caffe/blob/jason_public/doc/linux-no-root-install-log.md 使用服务器里的并行h5的话，相应的include和lib在/appl/opt/hdf5-par/GCC-4.9.3/intelmpi-5.1.1/1.8.15

jingang-cv commented 6 years ago

It seems to make no sense if includes the parallel version. Temporarily, just first import h5py instead.

mikecheninoulu commented 6 years ago

im curious who is the asiandragon