search

National-COVID-Cohort-Collaborative / Data-Ingestion-and-Harmonization

Data Ingestion and Harmonization
41 stars 12 forks source link

Palantir Data Characterization: Quantitative Data #62

Open hlehmann17 opened 3 years ago

hlehmann17 commented 3 years ago
Generalization: Normalizing Values In Measurement, Observation Where Implemented
For all numerical concept IDs [ OR set of given concept IDs] List of target concept ids   Concept ID white list 
Get frequency distributions of units   Data Quality Portal
...Unmapped, Mapped to 0, some other information
...Are we allowed to impute units? (from population? from other measurements within patient)
Determine significance of the distribution    
....Is there problematic data partner? Look for units = 0  
....Do different source data models (even tables within CDMs) behave differently? e.g., Obs_Clin
Articulate a formula for normalizing values and units    

Fill the (new) column, Normalized_value, Normalized_unit