search

kiyo-masui / bitshuffle

Filter for improving compression of typed binary data.
Other
215 stars 76 forks source link

error while running bitshuffle dynamic plugin #89

Closed SujitC closed 2 years ago

SujitC commented 3 years ago

Hello, I get the same error (https://github.com/h5py/h5py/issues/923#issue-256975657) when running sample code from 'h5ex_d_bshuf.c' with following environment:

HDF5: 1.12.0 (built with cmake) BitShuffle: a. master built with above HDF5 or b. release h5pl-1.12.0-win64 Visual Studio 2019, x64 Windows 10

Followed all the instruction given in https://portal.hdfgroup.org/display/support/HDF5+Filter+Plugins other plugin filters like lzf and lz4 are working as expected.

Here is the error stack for reference...

BitShuffle filter is available for encoding and decoding. ....Create dataset ................ HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:

000: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Pocpl.c line 1027 in H5Pget_filter_by_id2(): can't find object for ID

major: Object atom
minor: Unable to find atom information (already closed?)

001: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Pint.c line 4015 in H5P_object_verify(): property list is not a member of the class

major: Property lists
minor: Unable to register new atom

002: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Pint.c line 3965 in H5P_isa_class(): not a property list

major: Invalid arguments to routine
minor: Inappropriate type

HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:

000: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5D.c line 151 in H5Dcreate2(): unable to create dataset

major: Dataset
minor: Unable to initialize object

001: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5VLcallback.c line 1869 in H5VL_dataset_create(): dataset create failed

major: Virtual Object Layer
minor: Unable to create file

002: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5VLcallback.c line 1835 in H5VL__dataset_create(): dataset create failed

major: Virtual Object Layer
minor: Unable to create file

003: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5VLnative_dataset.c line 75 in H5VL__native_dataset_create(): unable to create dataset

major: Dataset
minor: Unable to initialize object

004: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Dint.c line 411 in H5D__create_named(): unable to create and link to dataset

major: Dataset
minor: Unable to initialize object

005: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5L.c line 1804 in H5L_link_object(): unable to create new link to object

major: Links
minor: Unable to initialize object

006: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5L.c line 2045 in H5L__create_real(): can't insert link

major: Links
minor: Unable to insert object

007: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Gtraverse.c line 855 in H5G_traverse(): internal path traversal failed

major: Symbol table
minor: Object not found

008: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Gtraverse.c line 630 in H5G__traverse_real(): traversal operator failed

major: Symbol table
minor: Callback failed

009: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5L.c line 1851 in H5L__link_cb(): unable to create object

major: Links
minor: Unable to initialize object

010: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Oint.c line 2522 in H5O_obj_create(): unable to open object

major: Object header
minor: Can't open object

011: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Doh.c line 301 in H5O__dset_create(): unable to create dataset

major: Dataset
minor: Unable to initialize object

012: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Dint.c line 1305 in H5D__create(): unable to set local filter parameters

major: Dataset
minor: Unable to initialize object

013: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Z.c line 935 in H5Z_set_local(): local filter parameters not set

major: Data filters
minor: Error from filter 'set local' callback

014: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Z.c line 864 in H5Z_prepare_prelude_callback_dcpl(): unable to apply filter

major: Data filters
minor: Error from filter 'can apply' callback

015: C:\Projects\HDF5\CMake-hdf5-1.12.0\hdf5-1.12.0\src\H5Z.c line 779 in H5Z_prelude_callback(): error during user callback

major: Data filters
minor: Error from filter 'set local' callback

failed to create dataset.

Kindly suggest the solution to make the bitshuffle algorithm work in my enviornment. Thanks.

mkitti commented 2 years ago

One way I ran into this problem was by having more than one HDF5 library (DLL) being involved. If you are using h5py, and you also followed the instructions from HDF Group, then you quite possibly have two copies of the HDF5 library being loaded (one from HDF Group and another via your h5py install).

The problem is that the property list gets created in one HDF5 library, but then the plugin calls back into another HDF5 library, which has no idea what property list id you are talking about because it did not create it.

jrs65 commented 2 years ago

I believe this will be fixed with the recent changes to dynamically load the libhdf5 library at run time. Let me know if you have any further issues.