[discussion] hdf file browser

[swelborn]

Hello! I stumbled across this package https://github.com/m-rossi/hdf5widget and it works very well to traverse hdf5 files in jupyter notebooks. May be of interest to you all - not to add to dxchange, but just FYI.

Working on a way to access hdf data interactively and plot it.

The read_hdf_meta function works great for showing the hdf contents via text, but I'm wondering if you have anything interactive. If any of you have a solution for this already, shoot it my way.

Here is a preview - goal here is to be able to click on a dataset and have it plot in the viewer to the right.

This is also a nice package to view and visualize data, but they do not have a python API for it so you cannot access the data you select (as far as I am aware): https://github.com/silx-kit/jupyterlab-h5web.

[decarlof]

meta is based on the same code used by read_hdf_meta(). It is very limited but you can show the tree, values + units and allows to change a value. I found it useful for quick checking of a variable value stored in multiple hdf files (using the --key option on a folder). For example:

meta show --file-name /data/ --key start_date

2022-08-09 19:20:26,462 -   *** file 0/107;  SpFn_T2_288_4_rman_001.h5
/process/acquisition/start_date 2022-07-13T10:12:52-0500
2022-08-09 19:20:26,575 -   *** file 1/107;  SpFn_T2_288_4_rman_take_2_002.h5
/process/acquisition/start_date 2022-07-13T10:35:09-0500
2022-08-09 19:20:26,683 -   *** file 2/107;  e6398_003.h5
/process/acquisition/start_date 2022-07-13T11:35:09-0500
....

meta show --file-name /data/ --key energy

2022-08-09 19:23:02,658 -   *** file 0/107;  SpFn_T2_288_4_rman_001.h5
/measurement/instrument/monochromator/energy 25.51 keV
2022-08-09 19:23:02,773 -   *** file 1/107;  SpFn_T2_288_4_rman_take_2_002.h5
/measurement/instrument/monochromator/energy 25.51 keV
2022-08-09 19:23:02,884 -   *** file 2/107;  e6398_003.h5
/measurement/instrument/monochromator/energy 25.51 keV
2022-08-09 19:23:02,994 -   *** file 3/107;  e6398_004.h5
/measurement/instrument/monochromator/energy 25.51 keV
2022-08-09 19:23:03,103 -   *** file 4/107;  e6398_005.h5
/measurement/instrument/monochromator/energy 25.51 keV

[prjemian]

https://github.com/nexpy/nexpy is a graphical interface to HDF5 files. Written for NeXus, it can render non-NeXus HDF5 content as well. Attached is a screen view of a single frame 1k,1k from EPICS area detector and a zip file with the HDF5 file. The GUI embeds a Python terminal as well.

On 8/9/2022 2:27 AM, swelborn wrote:

Hello! I stumbled across this package https://github.com/m-rossi/hdf5widget https://github.com/m-rossi/hdf5widget and it works very well to traverse hdf5 files in jupyter notebooks. May be of interest to you all - not to add to dxchange, but just FYI.

Working on a way to access hdf data interactively and plot it.

The read_hdf_meta function works great for showing the hdf contents via text, but I'm wondering if you have anything interactive. If any of you have a solution for this already, shoot it my way.

Here is a preview - goal here is to be able to click on a dataset and have it plot in the viewer to the right. image https://user-images.githubusercontent.com/55868530/183589323-78dfa5b8-9606-4880-897c-4552091f5b5c.png

This is also a nice package to view and visualize data, but they do not have a python API for it so you cannot access the data you select (as far as I am aware): https://github.com/silx-kit/jupyterlab-h5web https://github.com/silx-kit/jupyterlab-h5web.

— Reply to this email directly, view it on GitHub https://github.com/data-exchange/dxchange/issues/103, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARMUMCY7AIEXWLYNH4KAZ3VYIB6LANCNFSM557WL7NQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[prjemian]

PyMCA (http://pymca.sourceforge.net/) will also browse and display 1-D and 2-D data from HDF5 files. Screen view of same data attached. Not as easy as NeXPy (I had to press auto-scale dozens of times to get the full 1k,1k view - original view showed a 118,118 slice). No Python terminal interface.

On 8/9/2022 2:07 PM, Pete Jemian wrote:

https://github.com/nexpy/nexpy is a graphical interface to HDF5 files.  Written for NeXus, it can render non-NeXus HDF5 content as well.  Attached is a screen view of a single frame 1k,1k from EPICS area detector and a zip file with the HDF5 file.  The GUI embeds a Python terminal as well.

On 8/9/2022 2:27 AM, swelborn wrote:

Hello! I stumbled across this package https://github.com/m-rossi/hdf5widget https://github.com/m-rossi/hdf5widget and it works very well to traverse hdf5 files in jupyter notebooks. May be of interest to you all - not to add to dxchange, but just FYI.

Working on a way to access hdf data interactively and plot it.

The read_hdf_meta function works great for showing the hdf contents via text, but I'm wondering if you have anything interactive. If any of you have a solution for this already, shoot it my way.

Here is a preview - goal here is to be able to click on a dataset and have it plot in the viewer to the right. image https://user-images.githubusercontent.com/55868530/183589323-78dfa5b8-9606-4880-897c-4552091f5b5c.png

This is also a nice package to view and visualize data, but they do not have a python API for it so you cannot access the data you select (as far as I am aware): https://github.com/silx-kit/jupyterlab-h5web https://github.com/silx-kit/jupyterlab-h5web.

— Reply to this email directly, view it on GitHub https://github.com/data-exchange/dxchange/issues/103, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARMUMCY7AIEXWLYNH4KAZ3VYIB6LANCNFSM557WL7NQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[swelborn]

Thanks for this info, it's good to see what else is out there.

@prjemian - looks like it would work great for apps built with pyqt (is this right?), and I will keep it in mind if I work with that in the future. I'm trying to add interactive functionality to tomopyui, which is based around ipywidgets/housed within a jupyter notebook. I want to be able to first import my data with an hdf5 file explorer, and then use other functionality in my app afterwards.

@decarlof nice, I think it could be used for some kind of status bar in the application.

I have managed to figure something out here. It is not fully integrated into tomopyui yet, but it is a start:

https://user-images.githubusercontent.com/55868530/184015574-7405b3ab-1394-41b8-810c-da604b2c60b4.mp4

See recent commits to the branch here for details: https://github.com/swelborn/tomopyui/tree/multi_energy

It is pretty specific to the way tomopyui stores data (for the plotting aspect), but it could be made more general.