make_features exception --?empty FCS handling?

[hermands]

(flow_anal-env)hermands@yeti:~/working/Hodgkin/TESTS/01172015b$ /home/local/AMC/hermands/repos/flow_anal/dev/../flowanal.py -v make_features /net/emc/projects/dan_herman_project/FCS_Files/ -db /home/local/AMC/ngdavid/workspace/FCS_Full/fcs_meta_full.db --feature-hdf5 fcs_features.hdf5 --cases 14-16445 --tubes Hodgkin INFO make_features 54 Creating hdf5 file [fcs_features.hdf5] with features extracted by method [Full] INFO make_features 57 Loading database input /home/local/AMC/ngdavid/workspace/FCS_Full/fcs_meta_full.db INFO query_database 39 Querying... INFO query_database 358 Looking for db creation_date INFO query_database 39 Querying... INFO query_database 130 Looking for files INFO query_database 464 Looking for tubes: [u'Hodgkin'] INFO query_database 487 Looking for cases #14-16445 INFO make_features 71 Found 1 case_tube_idx's INFO make_features 76 Case: 14-16445, Case_tube_idx: 207419, File: 2014 archived flow files/14-16000-21999/14-16400-16499/14-16445/14-16445_Hodgkins.fcs [1 of 1] /home/local/AMC/hermands/repos/flow_anal/FlowAnal/FCS.py:64: UserWarning: loading FCS as empty because unpack requires a string argument of length 0 warnings.warn("loading FCS as empty because %s" % e) Skipping FCS /net/emc/projects/dan_herman_project/FCS_Files/2014 archived flow files/14-16000-21999/14-16400-16499/14-16445/14-16445_Hodgkins.fcs because of unknown error related to: <type 'exceptions.AttributeError'> [[u'14-16445', 207419, <type 'exceptions.AttributeError'>]]

[hermands]

(flow_anal-env)hermands@yeti:~/working/Hodgkin/TESTS/01172015b$ ls -lrth total 8.0K -rw-rw-r-- 1 hermands domain^users 5.0K Jan 17 10:28 fcs_features.hdf5

[hermands]

Meta db has it loaded as non-empty: sqlite> select * from TubeCases where case_number = '14-16445'; case_tube_idx|case_tube|filename|dirname|case_number|tube_type_instance|date|num_events|cytometer|cytnum|flag|error_message|version 207418|14-16445_T Cell|14-16445_T Cells.fcs|2014 archived flow files/14-16000-21999/14-16400-16499/14-16445|14-16445|3|2014-09-21 14:04:12|1624056|LSRII|2|GOOD||0.0.0.unknown 207419|14-16445_Hodgkin|14-16445_Hodgkins.fcs|2014 archived flow files/14-16000-21999/14-16400-16499/14-16445|14-16445|10|2014-09-21 14:04:17|1687556|LSRII|2|GOOD||0.0.0.unknown 207420|14-16445_B Cell|14-16445_B Cells.fcs|2014 archived flow files/14-16000-21999/14-16400-16499/14-16445|14-16445|4|2014-09-21 14:04:07|1435853|LSRII|2|GOOD||0.0.0.unknown

So, it is loading as good in metadb run, but now it is loading as Empty?

Is this because of something wrong with FCS 'data'????

Should initial meta db do something to test data portion? Or do we need to be able to handle this error here??

[hermands]

It had looked like make_feature failed to me, but it was because the counting doesn't include the exceptions.

• This should be fine for now -- these are failures to extract features and handled fine for the moment.