ERROR: test_nexus (pyFAI.test.test_io.TestSpotWriter.test_nexus) when run in debian12

[picca]

Hello Jerome,

I get these error messages when I run the unit test during the build

for information, I already installed the hdf5plugin package

====================================================================== ERROR: test_cxi (pyFAI.test.test_io.TestSpotWriter.test_cxi)

Traceback (most recent call last): File "/<>/.pybuild/cpython3_3.11_pyfai/build/pyFAI/test/test_io.py", line 270, in test_cxi io.spots.save_spots_cxi(tmpfile, self.spots, beamline="beamline", ai=self.ai) File "/<>/.pybuild/cpython3_3.11_pyfai/build/pyFAI/io/spots.py", line 211, in save_spots_cxi result.create_dataset("peakTotalIntensity", data=total_int, cmp) File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/group.py", line 161, in create_dataset dsid = dataset.make_new_dset(group, shape, dtype, data, name, kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/dataset.py", line 106, in make_new_dset dcpl = filters.fill_dcpl( ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/filters.py", line 281, in fill_dcpl raise ValueError("Unknown compression filter number: %s" % compression) ValueError: Unknown compression filter number: 32008

====================================================================== ERROR: test_nexus (pyFAI.test.test_io.TestSpotWriter.test_nexus)

Traceback (most recent call last): File "/<>/.pybuild/cpython3_3.11_pyfai/build/pyFAI/test/test_io.py", line 264, in test_nexus io.spots.save_spots_nexus(tmpfile, self.spots, beamline="beamline", ai=self.ai) File "/<>/.pybuild/cpython3_3.11_pyfai/build/pyFAI/io/spots.py", line 107, in save_spots_nexus peaks_grp.create_dataset("index", data=index, cmp) File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/group.py", line 161, in create_dataset dsid = dataset.make_new_dset(group, shape, dtype, data, name, kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/dataset.py", line 106, in make_new_dset dcpl = filters.fill_dcpl( ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/h5py/_debian_h5py_serial/_hl/filters.py", line 281, in fill_dcpl raise ValueError("Unknown compression filter number: %s" % compression) ValueError: Unknown compression filter number: 32008

Cheers

[picca]

The problematic filter is

bitshuffle

Filter Id: 32008

Filter Description:

This filter shuffles data at the bit level to improve compression. CHIME uses this filter for data acquisition.

Links:

[bitshuffle](https://github.com/kiyo-masui/bitshuffle)
[CHIME](http://chime.phas.ubc.ca/)

Contact Information:

Kiyoshi Masui
Email: kiyo at physics dot ubc dot ca

If I look at the python3-hdf5plugin package

$ apt show python3-hdf5plugin
Package: python3-hdf5plugin
Version: 4.0.1-3
Priority: optional
Section: python
Source: python-hdf5plugin
Maintainer: Debian Science Maintainers <debian-science-maintainers@lists.alioth.debian.org>
Installed-Size: 101 kB
Depends: python3:any, python3-h5py
Recommends: hdf5-filter-plugin-blosc-serial, hdf5-filter-plugin-bz2-serial, hdf5-filter-plugin-lz4-serial, hdf5-filter-plugin-zfp-serial, hdf5-plugin-lzf
Homepage: https://github.com/silx-kit/hdf5plugin
Download-Size: 22,9 kB
APT-Sources: http://deb.debian.org/debian unstable/main amd64 Packages
Description: Python library to make HDF5 compression filters usable from h5py
 hdf5plugin provides h5py extensions to control HDF5 compression filters
 plugins installed in the system.
 .
 The package supports providing options to:
 * blosc
 * bitshuffle
 * lz4
 * FCIDECOMP
 * ZFP
 * zstd
 .
 These filter plugins are not provided by this package and need to be installed
 using their own packages.

bitshuffle should be provided...

[picca]

Here the list of the files of the python3-hdf5plugin

/usr/lib/python3/dist-packages/hdf5plugin-4.0.1.egg-info/PKG-INFO
/usr/lib/python3/dist-packages/hdf5plugin-4.0.1.egg-info/dependency_links.txt
/usr/lib/python3/dist-packages/hdf5plugin-4.0.1.egg-info/not-zip-safe
/usr/lib/python3/dist-packages/hdf5plugin-4.0.1.egg-info/requires.txt
/usr/lib/python3/dist-packages/hdf5plugin-4.0.1.egg-info/top_level.txt
/usr/lib/python3/dist-packages/hdf5plugin/__init__.py
/usr/lib/python3/dist-packages/hdf5plugin/_config.py
/usr/lib/python3/dist-packages/hdf5plugin/_filters.py
/usr/lib/python3/dist-packages/hdf5plugin/_utils.py
/usr/lib/python3/dist-packages/hdf5plugin/_version.py
/usr/lib/python3/dist-packages/hdf5plugin/test.py
/usr/share/doc/python3-hdf5plugin/changelog.Debian.gz
/usr/share/doc/python3-hdf5plugin/changelog.gz
/usr/share/doc/python3-hdf5plugin/copyright

There is no hdf5plugin provided in this package..., the REcommands should be real dependencies.

[picca]

I will add the dependencies for now and see what is going on...

[picca]

with bitshuffle it PASS :))

[kif]

OK, with this dependencies, it is OK. Can I close the issue ?

[picca]

I think so, but I need to fix the hdf5plugin package.

Is is expected to provide all the real hdf5 implementation ?

[kif]

Yes, hdf5plugin is supposed to provide this ... but it had been splitted within debian and the bug is probably there.