NSLS-II / eiger-io

I/O tools for the Eiger detector
BSD 3-Clause "New" or "Revised" License
1 stars 9 forks source link

Fix handlers #8

Closed jrmlhermitte closed 6 years ago

jrmlhermitte commented 6 years ago

First pass at handlers. TODO : need to check with Yugang that this works with his analysis code (and doesn't run any slower...) I basically copied and pasted bits of the old handler for the PIMS version and wrote the dask version from scratch. Note that in the new version, EigerImages does not take filename as argument but rather dask array (we can do this since the array given by handler is already lazy).

What this does:

  1. Consolidates the specs into one handler. The only difference between AD_EIGER and AD_EIGER2 is the keyword argument supplied to the resource. It was frame_per_point for AD_EIGER and images_per_file in AD_EIGER2.

Normal way:

from eiger_io.fs_handler import EigerHandler
from databroker import Broker

db = Broker.named("chx")

db.reg.register_handler('AD_EIGER', EigerHandler, overwrite=True)
db.reg.register_handler('AD_EIGER2', EigerHandler, overwrite=True)

# test
uid = 'be6e4c'
h = db[uid]

events = list(h.data('eiger4m_single_image'))
# this is only a list of Pims arrays
events

Dask way:

from eiger_io.fs_handler_dask import EigerDaskHandler
from eiger_io.tools import dask_images

from databroker import Broker

db = Broker.named("chx")

db.reg.register_handler('AD_EIGER2', EigerDaskHandler, overwrite=True)
db.reg.register_handler('AD_EIGER2', EigerDaskHandler, overwrite=True)

# test
uid = 'be6e4c'
h = db[uid]
#list of dask arrays
events = list(h.data('eiger4m_single_image'))
events

# experimental: turn everything into a dask array
imgs = dask_images(h, 'eiger4m_single_image')
imgs

@danielballan @mrakitin

jrmlhermitte commented 6 years ago

Ran this test on all possible 4 file formats. I am able to retrieve images. Need to check with Yugang (warning this code is messy): https://gist.github.com/jrmlhermitte/750b031a74a9529535d7d25a6d7cb98f

danielballan commented 6 years ago

This seem like a good time to add some tests on realistic HDF5 files. In order to keep eiger-io lightweight, let's generate HDF5 files as part of the test process (rather than putting binary HDF5 files under version control).