search

silx-kit / pyFAI

Fast Azimuthal Integration in Python
Other
101 stars 93 forks source link

support for multi-frame hdf5 files #2221

Open marcocamma opened 1 week ago

marcocamma commented 1 week ago

Dear Jerome and all,

as you are certainly aware of, lima2 can handle more than one images taken with more than one threshold. At id10, we have already one Eiger2 4M running with Lima2 When I tried to use pyFAI-average, we get the error message reported below (pyFAI 2024.5.0)

It would be great if pyFAI-average (and possibly other CLIs?) can be improved to support Lima2 files. Thanks, marco

#####@#####:/data/visitor/hc5730/id10/20240611/RAW_DATA/yzo2/yzo2_0001/scan0122$ pyFAI-average -o sum.npy -m sum eiger4m_v2_frame_0_00000.h5
[ .....]
File "/usr/lib/python3/dist-packages/pyFAI/average.py", line 877, in process
writer.write_reduction(algorithm, image_reduction)
File "/usr/lib/python3/dist-packages/pyFAI/average.py", line 562, in write_reduction
image = self._fabio_class.__class__(data=data, header=header)
File "/usr/lib/python3/dist-packages/fabio/edfimage.py", line 801, in init
raise Exception("Data dimension too big. Only 1d or 2d arrays are supported.")
Exception: Data dimension too big. Only 1d or 2d arrays are supported.
kif commented 1 week ago

Apparently the file-driver used is currently EDF ... can you try to switch format with --format lima or numpy or eiger.

marcocamma commented 1 week ago

Indeed it works with --format numpy. Do you think that it could be possible to have the format be inferred if the -o filename is provided ? It would make the tool a bit handier to use (and it is already very handy!) Thanks anyway for your help !

kif commented 1 week ago

Well, I tried to understand the reason for the bug, and EDF is tailored for 1&2D data. Here the dataset looks of higher dimentionnality. The fact is: this is a bug which deserves a proper fix, and I suspected other fileformat would be more permissive. I need first to have a proper look at the data coming from those new detectors.

About the automatic determination of the fileformat: it is true a .npy extension should be associated to numpy but tiff or hdf5 have several fileformats associated with a single extension so a generic solution is not really achievable.