OHDSI / ETL-CMS

Workproducts to ETL CMS datasets into OMOP Common Data Model
Apache License 2.0
94 stars 52 forks source link

KeyError: 'Place of Service' #60

Closed galen211 closed 2 years ago

galen211 commented 5 years ago

I downloaded the SNOMED, ICD9CM, ICD9Proc, CPT4, HCPCS, LOINC, RxNorm, and NDC CDMv5 Vocabularies from Athena following the instructions in /python_etl/README.md (step 3). When I run a test of the ETL process using python CMS_SynPuf_ETL_CDM_v5.py 0, I get the following error after awhile:

Done, omop concept recs_in            = 35135870
recs_skipped                          = 31294019
len source_code_concept_dict           = 0
Reading omop_concept_file        -> /Volumes/hadrianus/Data/OHDSI/OMOP/02_Vocabulary/Standard_Vocabulary_v5/CONCEPT.csv
Writing to log file              -> /Volumes/hadrianus/Data/Evidera/98_output/concept_debug_log.txt
loaded domain dict with this many records:  15639
Traceback (most recent call last):
  File "CMS_SynPuf_ETL_CDM_v5.py", line 2028, in <module>
    build_maps()
  File "CMS_SynPuf_ETL_CDM_v5.py", line 464, in build_maps
    destination_file = domain_destination_file_list[domain_id]
KeyError: 'Place of Service'

Below is the referenced concept_debug_log.txt concept_debug_log.txt

galen211 commented 5 years ago

solution: add 'Place of Service' to domain_destination_file_list, line 227 in CMS_SynPuf_ETL_CDM_v5.py:

    'Place of Service'      : DESTINATION_FILE_VISIT
aaalgo commented 3 years ago

It seems the vocabulary is evolving. I got Key Error: 'Meas Value'.

Is the following patch appropriate?

-    'Visit'                 : DESTINATION_FILE_VISIT
+    'Visit'                 : DESTINATION_FILE_VISIT,
+    'Place of Service'      : DESTINATION_FILE_VISIT,
+    'Meas Value'            : DESTINATION_FILE_MEASUREMENT,

@galen211

Amseamle commented 2 years ago

How do you know which destination file e.g. condition, procedure, measurement, visit, etc a new key should be added to?