extracting onekey from pyCHX results file returns an empty dictionary

[ambarb]

https://github.com/NSLS-II/pyCHX/blob/6ae02935814caa0ca1efe413bc480a74c4c8fb94/pyCHX/Create_Report.py#L1826 Using this function with default arguments to extract data. Works well, but so much data takes a long time. I tried to speed up by only asking for what I want. I tried the follow:

• onekey argument
• excluded_keys argument

excluded_keys

this is the best hack around accessing just 1 key so you are not pulling all 2-time and roi intensity with time data if you just want an average image or g2b and tb. Did notice, that accessing 2time data (g12b ) is only 2 seconds on the server, but the gateway must be bogging down the communication to the notebook.

onekey

tried onekey = 'md', onekey = 'g12b', onekey = 'avg_img', onekey = 'imgsum' and all behaved the same way.

It seems that https://github.com/NSLS-II/pyCHX/blob/6ae02935814caa0ca1efe413bc480a74c4c8fb94/pyCHX/v2/_futurepyCHX/Create_Report.py#L2192 tried to implement the approach i used in my own editing of the function.

For my own function, the change i made doesn't seem to help though it seems to me to be exactly what was done in the debug function for the 'onekey' not None block. The debug function cannot extract md or g12b. Didn't try others.

For the the "_debug" version, not applying any extra arguments, there is an error so it seems this is also a work in progress. This is with default excluded_keys and default onekey

----> 2 total_res = load_data_from_pipeline_onekey(uid,)# onekey_s = 'md')#onekey='g12b')

/XF11ID/analysis/2018_3/lwiegart/development/Auto_Analysis_Functions_dev1.py in load_data_from_pipeline_onekey(uid, onekey_s)
     29                 #print('also tried: '+inDir+'uid=%s_Res.h5'%uid_full)
     30                 input_filename='uid=%s_Res.h5'%uid_full
---> 31                 total_res[u] = extract_xpcs_results_from_h5_debug( filename = 'uid=%s__Res.h5'%uid_full, import_dir = inDir,exclude_keys=exclude_keys, onekey = onekey_s )
     32 
     33         #total_res[uid] = extract_xpcs_results_from_h5( filename = 'uid=%s_phi_Res.h5'%uid_full, import_dir = inDir+ uid + '/' )

/XF11ID/analysis/2018_3/lwiegart/development/Auto_Analysis_Functions_dev1.py in extract_xpcs_results_from_h5_debug(filename, import_dir, onekey, exclude_keys)
     29                 if key not in exclude_keys:
     30                     if key in dicts:
---> 31                         extract_dict[key] = recursively_load_dict_contents_from_group(
     32                             hf, "/" + key + "/"
     33                         )

NameError: name 'recursively_load_dict_contents_from_group' is not defined

[ambarb]

Please join in the discussion on how to fix, but more importantly, the ultimate location of the extraction function for common use.

[ambarb]

Do we want to return a named tuple? Cannot return pandas because some arrays are 3D

[ambarb]

note to self or others new to this library. don't name a list of exlcuded_keys "get_avg_img" because this is a function in standard pyCHX imports to calculate the average images (which I guess is what generated the array found in the pyCHX results file.