KeyError: 'AD_EIGER2'

[yugangzhang]

Function:

def get_sid_filenames(header):
    """get a bluesky scan_id, unique_id, filename by giveing uid and detector

    Parameters
    ----------
    header: a header of a bluesky scan, e.g. db[-1]

    Returns
    -------
    scan_id: integer
    unique_id: string, a full string of a uid
    filename: sring

    Usuage:
    sid,uid, filenames   = get_sid_filenames(db[uid])

    """
    # return header
    filepaths = []
    db = header.db
    # get files from assets
    res_uids = db.get_resource_uids(header)
    for uid in res_uids:
        datum_gen = db.reg.datum_gen_given_resource(uid)
        datum_kwarg_gen = (datum['datum_kwargs'] for datum in
                           datum_gen)
        filepaths.extend(db.reg.get_file_list(uid, datum_kwarg_gen))
    return header.start['scan_id'],  header.start['uid'], filepaths

Usage:

uid = 'c8a1fb1f' 
from databroker import Broker
db_chx = Broker.named('chx')
sud = get_sid_filenames(db_chx[uid])

Error message:

--------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-86-10bcf6de4896> in <module>()
----> 1 sud = get_sid_filenames(db_chx[uid])
      2 #print ('scan_id, full-uid, data path are:  %s--%s--%s'%(sud[0], sud[1], sud[2][0] ))
      3 #start_time, stop_time = '2017-2-24  12:23:00', '2017-2-24  13:42:00'
      4 #sids, uids, fuids  = find_uids(start_time, stop_time)

<ipython-input-85-26fafd1e0599> in get_sid_filenames(header)
     25         datum_kwarg_gen = (datum['datum_kwargs'] for datum in
     26                            datum_gen)
---> 27         filepaths.extend(db.reg.get_file_list(uid, datum_kwarg_gen))
     28     return header.start['scan_id'],  header.start['uid'], filepaths

/opt/conda_envs/analysis-2018-1.0/lib/python3.6/site-packages/databroker/assets/base_registry.py in get_file_list(self, resource_or_uid, datum_kwarg_gen)
    277         actual_resource = self.resource_given_uid(resource_or_uid)
    278         return self._api.get_file_list(actual_resource, datum_kwarg_gen,
--> 279                                        self.get_spec_handler)
    280 
    281     def get_history(self, resource_uid):

/opt/conda_envs/analysis-2018-1.0/lib/python3.6/site-packages/databroker/assets/core.py in get_file_list(resource, datum_kwarg_gen, get_spec_handler)
    286     See the methods for moving and copying on the Registry object.
    287     """
--> 288     handler = get_spec_handler(resource['uid'])
    289     return handler.get_file_list(datum_kwarg_gen)

/opt/conda_envs/analysis-2018-1.0/lib/python3.6/site-packages/databroker/assets/base_registry.py in get_spec_handler(self, resource)
    248 
    249         spec = resource['spec']
--> 250         handler = self.handler_reg[spec]
    251 
    252         key = (str(resource['uid']), handler.__name__)

/opt/conda_envs/analysis-2018-1.0/lib/python3.6/collections/__init__.py in __getitem__(self, key)
    883             except KeyError:
    884                 pass
--> 885         return self.__missing__(key)            # support subclasses that define __missing__
    886 
    887     def get(self, key, default=None):

/opt/conda_envs/analysis-2018-1.0/lib/python3.6/collections/__init__.py in __missing__(self, key)
    875 
    876     def __missing__(self, key):
--> 877         raise KeyError(key)
    878 
    879     def __getitem__(self, key):

KeyError: 'AD_EIGER2'

[danielballan]

The immediate problem here is that no Handler has been registered for 'AD_EIGER2'. I think you know how to fix that --- db.reg.register_handler('AD_EIGER2', EigerHandlerOrWhatever). You might be wondering, "Why do I even need Handler for this? I just want the files!" A Resource may correspond to one file or to multiple files (such as a sequence of TIFF files). We rely on the Handler to provide the list of files via the method get_file_list. You can examples in this file.

So you need to:

1. Register a Hander for 'AD_EIGER2'
2. Ensure that that Handler implements the method get_file_list, which is optional in general but required if you want to use it in this way.

[mrakitin]

Did (1), worked fine, got stuck with (2):

In [13]: sud = get_sid_filenames(db_chx[uid])
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-13-f7a48aa637dc> in <module>()
----> 1 sud = get_sid_filenames(db_chx[uid])

<ipython-input-7-5e676201252f> in get_sid_filenames(header)
     29         datum_kwarg_gen = (datum['datum_kwargs'] for datum in
     30                            datum_gen)
---> 31         filepaths.extend(db.reg.get_file_list(uid, datum_kwarg_gen))
     32     return header.start['scan_id'],  header.start['uid'], filepaths

/opt/conda_envs/analysis-2018-1.0/lib/python3.6/site-packages/databroker/assets/base_registry.py in get_file_list(self, resource_or_uid, datum_kwarg_gen)
    277         actual_resource = self.resource_given_uid(resource_or_uid)
    278         return self._api.get_file_list(actual_resource, datum_kwarg_gen,
--> 279                                        self.get_spec_handler)
    280 
    281     def get_history(self, resource_uid):

/opt/conda_envs/analysis-2018-1.0/lib/python3.6/site-packages/databroker/assets/core.py in get_file_list(resource, datum_kwarg_gen, get_spec_handler)
    287     """
    288     handler = get_spec_handler(resource['uid'])
--> 289     return handler.get_file_list(datum_kwarg_gen)

AttributeError: 'EigerHandler' object has no attribute 'get_file_list'

@danielballan, @jrmlhermitte, what would you suggest?

[jrmlhermitte]

file handler needs get_file_list, ill add it now. i suspect this was part of pyCHX which I think defined its own handler with get_file_list. iwlll respond soon....

[jrmlhermitte]

@yugangzhang please let's test this tomorrow maybe? https://github.com/NSLS-II-CHX/eiger-io/pull/13

work on my machine, good night!

[yugangzhang]

Thanks! Will try tomorrow.