openPMD / openPMD-api

:floppy_disk: C++ & Python API for Scientific I/O
https://openpmd-api.readthedocs.io
GNU Lesser General Public License v3.0
134 stars 51 forks source link

HDF5: Remove Support for Old Versions #1514

Open ax3l opened 10 months ago

ax3l commented 10 months ago

If we bump up our support to newer HDF5 versions, then we can optimize meta-data features & performance:

The call H5Pset_libver_bounds will set the guaranteed compatibility with older readers. The less old versions are supported, the more performant we can be.

Cool features in Quincy Koziol's ATPESC23 or HUG23 slides (to be linked):

We got to check, but can probably dump everything before 1.12 because our users & readers use recent pip/conda packages that mostly ship HDF5 1.14 already.

cc @jeanbez already increased quite a few of these buffers, but I was unaware of H5Pset_libver_bounds performance implications so far.

jeanbez commented 10 months ago

@ax3l let me know the minimum version you plan to support so I can open a PR with this. I can check the other optimizations since most have already been applied. I'm curious about their possible impact on openPMD.

ax3l commented 3 weeks ago

@jeanbez I think we want to drop at least everything older than HDF5 1.10.

I would like to bump to 1.12+ in the comings years, too, but some distros like Ubuntu is still a bit behind: https://repology.org/project/hdf5/versions

jeanbez commented 3 weeks ago

Got it, I believe that's a good way to go! I'll take a look into that and at the changes to include the H5Pset_libver_bounds