search

kiyo-masui / bitshuffle

Filter for improving compression of typed binary data.
Other
219 stars 76 forks source link

Bitshuffle with Anaconda importation problem - Library not loaded #16

Closed ranaivomahaleo closed 3 years ago

ranaivomahaleo commented 9 years ago

I have the following problem when using bitshuffle with Anaconda. The following importation from bitshuffle import h5 returns the following error.

.../anaconda/lib/python2.7/site-packages/bitshuffle/h5.so, 2): Library not loaded: libhdf5.8.dylib
  Referenced from: .../anaconda/lib/python2.7/site-packages/bitshuffle/h5.so
  Reason: image not found

bitshuffle was correctly installed from git source.

My system:

Do you have a quick fix for it?

kiyo-masui commented 9 years ago

Does import h5py work in an interpreter?

ranaivomahaleo commented 9 years ago

I have an error when using this command (import h5py). See as follows

import h5py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "h5py/__init__.py", line 10, in <module>
    from h5py import _errors
ImportError: dlopen(h5py/_errors.so, 2): Library not loaded: @loader_path/../../../libhdf5.8.dylib
  Referenced from:.../anaconda/lib/python2.7/site-packages/h5py/_errors.so
  Reason: image not found
kiyo-masui commented 9 years ago

Okay, this is a problem with your environment, probably not with bitshuffle. Maybe try upgrading anaconda?

ranaivomahaleo commented 9 years ago

Conda can not manage correctly unregistered packages such as bitshuffle. To fix this issue (from bitshuffle import h5 error), I proceeded as follows:

simongregorebner commented 8 years ago

Installing bitshuffle via pip into a anaconda installation indeed cause problems in regard of its binary dependencies to hdf5. Have you tried the conda recipe within this repository to build and install this package as an anaconda package? This will just work fine without modifying any environment variables etc. .

nritsche commented 3 years ago

I also encountered this problem when trying to set up a macOS CI build on github actions (https://github.com/kiyo-masui/bitshuffle/pull/93/checks?check_run_id=2091445788)

pip install -r requirements.txtCollecting h5py>=2.4.0
  Downloading h5py-3.1.0.tar.gz (371 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting cached-property
  Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Building wheels for collected packages: h5py
  Building wheel for h5py (PEP 517): started
  Building wheel for h5py (PEP 517): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /Users/runner/hostedtoolcache/Python/3.6.13/x64/bin/python /Users/runner/hostedtoolcache/Python/3.6.13/x64/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmph8525wkb
Failed to build h5py
       cwd: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip-install-awfsn05o/h5py_3f392e001f8641eba9fa0f4e6621a26c
  Complete output (70 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.14-x86_64-3.6
  creating build/lib.macosx-10.14-x86_64-3.6/h5py
  copying h5py/h5py_warnings.py -> build/lib.macosx-10.14-x86_64-3.6/h5py
  copying h5py/version.py -> build/lib.macosx-10.14-x86_64-3.6/h5py
  copying h5py/__init__.py -> build/lib.macosx-10.14-x86_64-3.6/h5py
  copying h5py/ipy_completer.py -> build/lib.macosx-10.14-x86_64-3.6/h5py
  creating build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/files.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/compat.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/__init__.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/selections.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/dataset.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/vds.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/selections2.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/group.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/datatype.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/attrs.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/dims.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/base.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  copying h5py/_hl/filters.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/_hl
  creating build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_dimension_scales.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_attribute_create.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_file_image.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/conftest.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_h5d_direct_chunk.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_h5f.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_dataset_getitem.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_group.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_errors.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_dataset_swmr.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_slicing.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_h5pl.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_attrs.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/__init__.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_attrs_data.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_h5t.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_big_endian_file.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_h5p.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_dims_dimensionproxy.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_datatype.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/common.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_dataset.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_file.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_selections.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_dtype.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_h5.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_file2.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_completions.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_filters.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_base.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  copying h5py/tests/test_objects.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests
  creating build/lib.macosx-10.14-x86_64-3.6/h5py/tests/data_files
  copying h5py/tests/data_files/__init__.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/data_files
  creating build/lib.macosx-10.14-x86_64-3.6/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_highlevel_vds.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_virtual_source.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/test_vds
  copying h5py/tests/test_vds/__init__.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_lowlevel_vds.py -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/test_vds
  copying h5py/tests/data_files/vlen_string_s390x.h5 -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/data_files
  copying h5py/tests/data_files/vlen_string_dset_utc.h5 -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/data_files
  copying h5py/tests/data_files/vlen_string_dset.h5 -> build/lib.macosx-10.14-x86_64-3.6/h5py/tests/data_files
  running build_ext
  Loading library to get version: libhdf5.dylib
  error: Unable to load dependency HDF5, make sure HDF5 is installed properly
  error: dlopen(libhdf5.dylib, 6): image not found
  ----------------------------------------
  ERROR: Failed building wheel for h5py
ERROR: Could not build wheels for h5py which use PEP 517 and cannot be installed directly
nritsche commented 3 years ago

This is obviously not a problem with bitshuffle but with HDF5/h5py installations.