IOError for EigerImage.getframe() when loading last frame in container

[femi86]

when using fabio to unpack images from the dectris eiger proprietory hdf5 container using the eigerimage.py module, an IOError is raised in line 191. this is because the iter method of fabioimage.py calls the .next() method from eigerimage.py, but when the getframe() method is called on the last frame +1 in this method, the else condition in the getframe() method is invoked and raises the IOError("getframe %s out of range [%s %s[" % (num, 0, self.nframes)).

is there a reason for this else statement to exist at all? how can this be fixed?

[vallsv]

Hi. Could you paste the full exception? With few lines to reproduce the problem? I am not sure which API you use and the way you use it (in a for loop? or what).

Anyway, it sounds like we don't have test here, cause the use case looks very basic.

The reason is i guess for legacy. Difficult to know.

[femi86]

Here is the code I run:

import hdf5plugin # needed here for fabio
import h5py # needed here for fabio
import os,sys
import numpy as np
import PIL
from PIL import Image
import pyFAI
import fabio
import cv2

image_series = fabio.open(r'M:\Calibration_01\test_500ms_3_master.h5')
maskoff = fabio.open(r'M:\mask-off_v00_NoEdges.edf')
poni = r'M:\AVG_calibration_La-b6_30s.poni'

ai_poni = pyFAI.load(poni)
maskoff
theta_range = 4334
errors = {}

xrd_patterns = np.empty((image_series.nframes+1,theta_range))

if image_series.nframes > 1:
    for i,iframe in enumerate(image_series):
        res = ai_poni.integrate1d(iframe.data,
                                     theta_range,
                                     method='csr', 
                                     unit="2th_deg",
                                     mask=maskoff.data)

And here is the traceback error I receive:

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
C:\WPy64-3770\python-3.7.7.amd64\lib\site-packages\fabio\fabioimage.py in __iter__(self)
    853             try:
--> 854                 current_image = current_image.next()
    855             except IOError:

C:\WPy64-3770\python-3.7.7.amd64\lib\site-packages\fabio\eigerimage.py in next(self)
    207         """ returns the next frame in the series as a fabioimage """
--> 208         return self.getframe(self.currentframe + 1)
    209 

C:\WPy64-3770\python-3.7.7.amd64\lib\site-packages\fabio\eigerimage.py in getframe(self, num)
    195 #                       .format(num, 0, self.nframes))
--> 196                 raise IOError("getframe %s out of range [%s %s[" % (num, 0, self.nframes))
    197 

OSError: getframe 8 out of range [0 8[

During handling of the above exception, another exception occurred:

StopIteration                             Traceback (most recent call last)
C:\WPy64-3770\python-3.7.7.amd64\lib\site-packages\fabio\fabioimage.py in __iter__(self)
    855             except IOError:
--> 856                 raise StopIteration
    857 

StopIteration: 

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
<ipython-input-3-586aeb2688e8> in <module>
     22 
     23 if image_series.nframes > 1:
---> 24     for i,iframe in enumerate(image_series):
     25         res = ai_poni.integrate1d(iframe.data,
     26                                      theta_range,

RuntimeError: generator raised StopIteration

The datafile used is too large to be uploaded, but it is generated by a dectris Eiger detector, where it uses a proprietary algorithm to compress 2d images and header into an hdf5 container that is split between a master file and several subordinate datafiles each containing up to 500 images (I believe fabio uses the Albula API produced by dectris, but I am not sure about this, it goes beyond my knowledge)

[vallsv]

Could you replace your loop by that instead:

for iframe in image_series.frames():
    iframe.data # <-- should provide to you the data
    iframe.index # <-- should provide to you the 0, 1... your enumerate is fine too

This API is newer, so i expect it to be safer. I am afraid the other one is broken.

[vallsv]

On side note, i think this raise StopIteration should be removed (from the fabio code), and this also should fix your problem. But the implementation uses it. So there is maybe a real reason here, or it's just a crappy code with side effect.

[jonwright]

Long in the past, the first use case (I think GEimage) had blocks of frames spread over several files. People wanted to iterate over a series going from the end of one file to the beginning of the next.