All Of Us Data Quality Check on MU OMOP CDM - Githubissues

Missouri-BMI / OMOP_ON_P_CDM

Apache License 2.0

0 stars 0 forks source link

All Of Us Data Quality Check on MU OMOP CDM #41

Closed shossain-mizzou closed 9 months ago

vasanthi014 commented 11 months ago

@shossain-mizzou Did you work on this? Do you need anything to move forward?

shossain-mizzou commented 11 months ago

We need OMOP data first. I have tested with the test dataset

vasanthi014 commented 11 months ago

OMOP views are accessible by you. You can download it to a SWB and run the DQ program on it.

shossain-mizzou commented 9 months ago

@vasanthi014 , columns needs to be modified in visit_occurrence:

ADMITTED_FROM_CONCEPT_ID as admitting_source_concept_id
ADMITTED_FROM_SOURCE_VALUE as admitting_source_value
DISCHARGED_TO_CONCEPT_ID as discharge_to_concept_id
DISCHARGED_TO_SOURCE_VALUE as discharge_to_source_value

shossain-mizzou commented 9 months ago

[x] care_site.csv
[x] cdm_source.csv
[ ] condition_era
[x] condition_occurrence
[x] death
[ ] device_exposure
[ ] dose_era
[ ] drug_era
[x] drug_exposure
[ ] fact_relationship
[x] location
[x] measurement
[ ] note
[x] observation_period
[ ] observation
[ ] participant_match
[x] person
[x] procedure_occurrence
[ ] provider
[ ] relationship
[ ] source_concept_map
[ ] specimen
[ ] visit_detail
[x] visit_occurrence

shossain-mizzou commented 9 months ago

Screenshot 2023-12-05 at 2 14 15 PM

vasanthi014 commented 9 months ago

All these issues should be fixed now. I would expect NULL's in other columns, I will be revising the ETL to make sure the not NULL columns are addressed but please go ahead and run a report again.

shossain-mizzou commented 9 months ago

Ran on 1000 records per csv. Running on large dataset was talking longer time. We should deal with all the null values first.

Screenshot 2023-12-12 at 12 45 13 PM

shossain-mizzou commented 9 months ago

closing the ticket, we have to check null values though