Open LauZanMo opened 4 months ago
I spent some time clarifying the structure of PropertyList in HighFive and found it very easy to add filter on this basis. First, I need to declare and implement the Blosc class in H5PropertyList.hpp and H5PropertyList_misc.hpp, respectively. Just like this:
# H5PropertyList.hpp
/// \implements PropertyInterface
class Blosc {
public:
explicit Blosc(unsigned level, unsigned shuffle, unsigned compressor);
private:
friend DataSetCreateProps;
void apply(hid_t hid) const;
unsigned _cd_values[7];
};
# H5PropertyList_misc.hpp
inline Blosc::Blosc(unsigned int level, unsigned int shuffle, unsigned int compressor) {
_cd_values[4] = level;
_cd_values[5] = shuffle;
_cd_values[6] = compressor;
}
inline void Blosc::apply(const hid_t hid) const {
detail::h5p_set_blosc(hid, _cd_values);
}
Then, as for the specific implementation function ‘detail::h5p_set_blosc’, I refer to the implementation of hdf5_blosc: #https://github.com/Blosc/hdf5-blosc/blob/master/src/example.c#L57 just like this:
# h5p_wrapper.hpp
#include "blosc_filter.h"
inline herr_t h5p_set_blosc(hid_t plist_id, const unsigned int *cd_values) {
char *version, *date;
if (register_blosc(&version, &date) < 0) {
HDF5ErrMapper::ToException<PropertyException>("Blosc filter unavailable.");
}
herr_t err = H5Pset_filter(plist_id, FILTER_BLOSC, H5Z_FLAG_OPTIONAL, 7, cd_values);
if (err < 0) {
HDF5ErrMapper::ToException<PropertyException>("Error setting blosc property");
}
return err;
}
Of course I need to link hdf5-blosc dynamic library files:
# root CMakeLists.txt
# I install hdf5-blosc to the default path
set(blosc_filter_DIR /usr/local/lib/)
target_link_libraries(HighFive INTERFACE blosc_filter)
target_link_directories(HighFive INTERFACE /usr/local/include/)
Finally I can write with blosc-filter!
HighFive::DataSetCreateProps props;
props.add(HighFive::Chunking({4,4}));
props.add(HighFive::Blosc(4, 1, BLOSC_BLOSCLZ));
// create dataset and write with this properties!
Do you think you can propose a PR with that? It looks exactly what you should do to add your own filter.
OK, I'll submit a PR soon.
I would like to ask whether there is a way to change the default filter of highfive to blosc, because the default filter,which is set to szip, is slow and my system has real-time requirements. I notice that the highfive code does not seem to support the setting of filter. Can it be implemented using the native HDF5 function ?