ARCH-commons / i2p-transform

i2b2 to PCORnet Common Data Model Transformation - requires i2b2 PCORnet ontology
Other
3 stars 16 forks source link

only patients active since 2010? #4

Closed dckc closed 8 years ago

dckc commented 8 years ago

Why limit the CDM ETL to patients active since 2010?

@njgraham, I wonder if this constraint is applicable to us.

cc @mprittie, @rwaitman

select DISTINCT PATIENT_NUM from I2B2FACT where START_DATE > to_date('01-Jan-2010','dd-mon-rrrr')

https://github.com/SCILHS/i2p-transform/blob/master/Oracle/PCORNetLoader_ora.sql#L70

jklann commented 8 years ago

Agreed that this is not necessarily applicable to GPC. SCILHS has decided to standardize the start date of 1-1-10. Note also that the loyalty cohort finder has hard-coded dates at the moment:

https://github.com/SCILHS/SCILHS-utils/blob/master/LoyaltyCohort/LoyaltyCohort-ora.sql#L225

dckc commented 8 years ago

Our approach to this is to parameterize min_pat_list_date_dd_mon_rrrr.

I'm trying to isolate it as its own pull request, but I'm not having much luck.

Meanwhile, see https://github.com/kumc-bmi/i2p-transform/commit/3eee53e9714058b4030858d829ce8b3aeee3573f

dckc commented 8 years ago

The 2010 timeframe from i2p-transform doesn't seem to apply to the DEMOGRAPHIC table. We have just as many pcornet DEMOGRAPHIC records as we have i2b2 patient_dimension records. Is that by design? Or did we goof somehow?

cc @njgraham @rwaitman @mprittie

p.s. I found rationale for the 2010 cut-off in some PCORnet requirements somewhere, so we are sticking to it as well. Here's hoping I manage to cite them sometime.

jklann commented 8 years ago

Hmm, search for "i2b2patient_list" in the transform. You shouldn't get all of the patient_dimension, just those with records since 1/1/2010.

matthewjoss commented 8 years ago

I added the oracle label, even though I think this also still exists in the MSSQL code

jklann commented 8 years ago

Demographic now pulls only patients since 1-1-2010. The parameterized run script allows changed date constraints in MSSQL; will implement in Oracle.