NeXus HDF5 format support

OpenChemistry / tomviz

Cross platform, open source application for the processing, visualization, and analysis of 3D tomography data

https://tomviz.org/

BSD 3-Clause "New" or "Revised" License

317 stars 84 forks source link

NeXus HDF5 format support #1597

Open magnunor opened 6 years ago

magnunor commented 6 years ago

NeXus is a file format based on HDF5 for neutron, x-ray, muon science, and a definition for scanning transmission electron microscopy (STEM) is in the works: https://github.com/LiberTEM/LiberTEM/issues/15

A python library for interacting with the NeXus file format already exist, so it shouldn't be too hard to add a reader: https://github.com/nexpy/nexusformat

The initial STEM definition is for pixelated STEM, so is maybe not that applicable for tomviz, but extending the definition to tomography is likely to happen (@uellue might have some input on this).

cryos commented 6 years ago

I have been tracking this, the Python HDF5 module is tougher to package with Tomviz than expected, and it the primary blocker for using a Python library to support NeXus. I would like to get support for the format into Tomviz.

sk1p commented 6 years ago

There are binary wheels for h5py available if that helps, and the nexusformat package only depends on numpy and h5py as far as I can see. Otherwise the question would be: what can be done to ease the packaging?

Maybe the NeXus guys can also help with that? Generally the process of getting nexus format support into a new project should be made as painless as possible, of course, so knowing your pain points would help!

sk1p commented 6 years ago

cc @uellue

cryos commented 6 years ago

We already have SciPy and NumPy in the Tomviz package for all three platforms, and we recently added Python based readers and writers. The HDF5 library is packaged, and used from C++. It is pulling HDF5 out into our top-level packaging code, we really need to have the same HDF5 library from the C++ and Python side available with h5py.

mathturtle commented 6 years ago

we really need to have the same HDF5 library from the C++ and Python side available with h5py.

So this would include:

pull out HDF5
make ParaView use external HDF5 (paraview superbuild already does this, not too difficult)
fix ITK: either update to a version of ITK that properly mangles its HDF5 or make ITK not enable whatever needs HDF5 (Technically we could update the superbuild to make ITK depend on the external HDF5, but then ITK would need to be built by the superbuild rather than using prebuilt binaries)