search

MMU-TDMLab / ClinicalNTS

MIT License
6 stars 3 forks source link

GetDataFromMimic array out of bound errors! #4

Open vanh17 opened 3 years ago

vanh17 commented 3 years ago

Hi @mattshardlow and @ash-williams,

According to the paper, the code here used MIMIC-III version 1.4, however, when I used the same MIMIC-III version with this java code, it threw out-of-bound errors at two places:

  1. https://github.com/MMU-TDMLab/ClinicalNTS/blob/8b52044603d0bd86fb67cb987a6ed04f56274a94/DataCollection/MIMIC/GetDataFromMIMIC.java#L48
  2. https://github.com/MMU-TDMLab/ClinicalNTS/blob/8b52044603d0bd86fb67cb987a6ed04f56274a94/DataCollection/MIMIC/GetDataFromMIMIC.java#L50 For the first error, looks like some of the lines do not have the 12th element after the regex matching. For the second error, the system cannot ignore empty entries and hence throw errors at the run time.

I tried to put in check system in place and ignore lines/files that do not satisfy the two conditions above, i.e. need to have at least 12 elements (index starts at 0) and ignore empty lines. However, all it returned to me were texts from PRESCRIPTION.csv and these texts were just numbers.

Also, when I looked at the code here, the MIMIC-III files are expected to be in .txt format but the downloaded 1.4 version contains only .csv files. Because of this, I was just wondering if you used another preprocessor to convert .csv files into .txt format.

Thank you, @mattshardlow and @ash-williams!

ash-williams commented 3 years ago

Hi, I wasn't a part of this piece of work, but I will message Matt to make sure he sees it.

Cheers, Ash

vanh17 commented 3 years ago

@mattshardlow, another follow-up with the open issue.

I looked at the current MIMIC version here. This one is the most current one and it was released in 2016, which I suspect it was the one used in your paper. However, not all MIMIC .csv files have 11 columns or more. Did it give you any errors during the project or there might be a sublist of the MIMIC dataset that you passed onto the processing script at run-time?