search

bluesky / area-detector-handlers

https://pypi.org/project/area-detector-handlers
Other
1 stars 11 forks source link

Reraise h5py KeyError as IOError. #13

Closed danielballan closed 4 years ago

danielballan commented 4 years ago

Per h5py/HDF5 expert @tacaswell, it is difficult to h5py to handle all HDF5 errors and reraise them correctly because they are not treated as stable.

We encountered a situation at FXI where a file was not ready to be read, raised a KeyError, and caused the Filler to give up. We want the Filler to engage is retry-with-backoff-before-giving-up behavior.

We think it is correct that Filler only catches IOError in its backoff loop. Other exceptions could indicate real errors that should be reported immediately without waiting for a retry loop. It is the job of the handlers to ensure that they raise IOError when the underlying I/O library may be encountering an I/O issue that could be resolved by waiting and retrying.

danielballan commented 4 years ago

I am not sure how to go about unit testing this [because it arises from tricky race conditions inside libhdf5] but it worked in production at FXI.

danielballan commented 4 years ago

Thanks, @mrakitin. Not sure why I didn't see that. Refactored.

danielballan commented 4 years ago

I tacked on an unrelated commit to address a DeprecationWarning from h5py from one of the test fixtures.