search

MIT-LCP / critical-data-book

Code repository for the critical data book
MIT License
73 stars 52 forks source link

Chapter 18 (trend analysis), how to retrieve year? #14

Closed jinjinzhou closed 5 years ago

jinjinzhou commented 5 years ago

In the chartevents.csv table we downloaded,

one CHARTTIME, is 5/12/34 12:00

Can someone help with what does 34 mean? How to translate to 34 to a correct calendar year.

Thanks

Jin

jraffa commented 5 years ago

If you're extracting this from MIMIC II or III, the dates have been shifted, and the years should be nonsensical as the shifting is necessary to protect PHI. It is possible to apply for access to get the actual years, and I would e-mail mimic-support@physionet.org to ask how.

I am not certain, but the authors likely applied for this additional access themselves, and this was how they were able to write their chapter.

Since this isn't a question related to the code, I will close the issue, but if I have misunderstood, feel free to let me know.

tompollard commented 5 years ago

@jinjinzhou as Jesse mentions, the dates are shifted by several hundred years for de-identification purposes. Are you looking at the raw text or using an application like Excel? Excel mashes up the shifted dates, which might explain why you are seeing a nonsensical value.

jinjinzhou commented 5 years ago

Thanks, Jesse and Tom. We are using MIMIC-III. I am ccing to see if any algorithm we may have to translate year.

Thanks,

Jin

tompollard commented 5 years ago

@jinjinzhou There is no need to email us. We prefer to discuss these issues on the public repository (as noted at: https://mimic.physionet.org/help/).

You can typically treat the dates in MIMIC as standard dates. It should not be necessary to translate them. What software are you using the view the data?

jinjinzhou commented 5 years ago

Thanks, Tom. I followed instruction as noted in MIMIC-III to a postgres SQL database and vied them by R.

Jin

tompollard commented 5 years ago

The date that you've listed above (CHARTTIME, is 5/12/34 12:00) does not occur in MIMIC-III as far as I am aware. If this date appears in your database, please provide us with a query to find this value (e.g. select charttime from chartevents where row_id = X). Please also read https://www.nature.com/articles/sdata201635 and https://mimic.physionet.org/mimicdata/time/ if you have not already done so.

jraffa commented 5 years ago

How are you viewing it in R? Are you using RPostgreSQL?

jinjinzhou commented 5 years ago

Yes

jraffa commented 5 years ago

It is difficult to diagnosis this if we can't replicate it on our end. Are you are able to find what @tompollard has asked for? (row_id)

jinjinzhou commented 5 years ago

Here it looks like.

Jin

[cid:image001.png@01D488AD.8181E380]

From: Jesse Raffa notifications@github.com Reply-To: MIT-LCP/critical-data-book reply@reply.github.com Date: Friday, November 30, 2018 at 1:01 PM To: MIT-LCP/critical-data-book critical-data-book@noreply.github.com Cc: "Zhou, Jin - (jzhou)" jzhou@email.arizona.edu, Mention mention@noreply.github.com Subject: Re: [MIT-LCP/critical-data-book] Chapter 18 (trend analysis), how to retrieve year? (#14)

It is difficult to diagnosis this if we can't replicate it on our end. Are you are able to find what @tompollardhttps://github.com/tompollard has asked for? (row_id)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/MIT-LCP/critical-data-book/issues/14#issuecomment-443322699, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ANJYT4yzi6DjGF5DMeC1Yxv-1DAUkGSZks5u0Y6pgaJpZM4Y8LTl.

jraffa commented 5 years ago

Jin, can you post the image on github.com, it's not showing up here. https://github.com/MIT-LCP/critical-data-book/issues/14

jinjinzhou commented 5 years ago
screen shot 2018-11-30 at 12 58 02 pm
jinjinzhou commented 5 years ago
screen shot 2018-11-30 at 2 55 44 pm
MizzleChen commented 5 years ago

Jin, can you post the image on github.com, it's not showing up here.

14

The data set description about chartevents.csv is available from the website https://mimic.physionet.org/mimictables/chartevents/ The dates are all shifted based on the website https://mimic.physionet.org/mimicdata/time/ It seems that only year had been shifted. Month and day are from original record. I am just wondering is there anyway to find original year for each observation.

jinjinzhou commented 5 years ago

To echo above comment, we don't need to have exact year - just a number in the same order as the calendar year order. Thanks,

tompollard commented 5 years ago

The year (or sequence by year) is not available in MIMIC-III unfortunately. We understand that this is a limitation and are considering alternative options for handling de-identification for the next release of MIMIC.