Closed stephanieshong closed 2 years ago
These are fine.
Elevated using Concept ID: 3445772 is a non-standard vocabulary (Nebraska Lexicon). That source concept maps to the Concept ID: 4328749 (https://athena.ohdsi.org/search-terms/terms/4328749) where High replaces Elevated (https://athena.ohdsi.org/search-terms/terms/40287622) as of 01/01/2013.
Thank you. @kmkostka Does this mean other values are ok?
LAB_RESULT_CM | RESULT_QUAL | ABNORMAL | Abnormal | See notes | Gap |
---|---|---|---|---|---|
LAB_RESULT_CM | RESULT_QUAL | BORDERLINE | Borderline | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | ELEVATED | Elevated | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | HIGH | High | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | LOW | Low | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | NEGATIVE | Negative | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | NI | No Information | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | NORMAL | Normal | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | OT | Other | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | POSITIVE | Positive | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | UN | Unknown | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | UNDETECTABLE | Undetectable | See notes | Gap |
LAB_RESULT_CM | RESULT_QUAL | UNDETERMINED | Undetermined | See notes | Gap |
What concept id should we use for undetectable and undetermined?
Hi,
In her review, Kristen suggested using concept_id zero.
Thanks, Charles
From: Stephanie Hong notifications@github.com Sent: Monday, July 13, 2020 11:54 AM To: National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization Data-Ingestion-and-Harmonization@noreply.github.com Cc: Charles Yaghmour cyaghmour@samvit-solutions.com; Assign assign@noreply.github.com Subject: Re: [National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization] No qualifier concept ids defined in the ValueSet Mapping document (#55)
LAB_RESULT_CM RESULT_QUAL ABNORMAL Abnormal See notes Gap LAB_RESULT_CM RESULT_QUAL BORDERLINE Borderline See notes Gap LAB_RESULT_CM RESULT_QUAL ELEVATED Elevated See notes Gap LAB_RESULT_CM RESULT_QUAL HIGH High See notes Gap LAB_RESULT_CM RESULT_QUAL LOW Low See notes Gap LAB_RESULT_CM RESULT_QUAL NEGATIVE Negative See notes Gap LAB_RESULT_CM RESULT_QUAL NI No Information See notes Gap LAB_RESULT_CM RESULT_QUAL NORMAL Normal See notes Gap LAB_RESULT_CM RESULT_QUAL OT Other See notes Gap LAB_RESULT_CM RESULT_QUAL POSITIVE Positive See notes Gap LAB_RESULT_CM RESULT_QUAL UN Unknown See notes Gap LAB_RESULT_CM RESULT_QUAL UNDETECTABLE Undetectable See notes Gap LAB_RESULT_CM RESULT_QUAL UNDETERMINED Undetermined See notes Gap
What concept id should we use for undetectable and undetermined?
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization/issues/55#issuecomment-657641915, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGFG2JR3M6P4TALP7W6Y4J3R3MUZTANCNFSM4OJS4TQQ.
@cyaghmour - could you double confirm, and validate the following numbers?
when lower(trim(result_qual)) = 'un' then 45880480 else 45877393<--- default value ( all other cases INVALID, Not Detected, See Comments, TNP, QNS)
RESULT_QUAL | ABNORMAL | Abnormal | 45878745 |
---|---|---|---|
RESULT_QUAL | BORDERLINE | Borderline | 45880922 |
RESULT_QUAL | ELEVATED | Elevated | 4328749 |
RESULT_QUAL | HIGH | High | 45876384 |
RESULT_QUAL | LOW | Low | 45881666 |
RESULT_QUAL | NEGATIVE | Negative | 45878583 |
RESULT_QUAL | NI | No Information | 46237210 |
RESULT_QUAL | NORMAL | Normal | 45884153 |
RESULT_QUAL | OT | Other | 46237210 |
RESULT_QUAL | POSITIVE | Positive | 45884084 |
RESULT_QUAL | UN | Unknown | 45877986 |
RESULT_QUAL | UNDETECTABLE | Undetectable | 0 |
RESULT_QUAL | UNDETERMINED | Undetermined | 45880649 |
RESULT_QUAL | pos | pos | 45884084 |
RESULT_QUAL | neg | neg | 45878583 |
RESULT_QUAL | presumptive positive | presumptive positive | 45884084 |
RESULT_QUAL | presumptive negative | presumptive negative | 45878583 |
RESULT_QUAL | detected | detected | 45884084 |
RESULT_QUAL | not detected | not detected | 45878583 |
RESULT_QUAL | inconclusive | inconclusive | 45877990 |
https://github.com/National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization/issues/55
Could you double confirm concept Id that I populaed on this page?
Stephanie
From: cyaghmour notifications@github.com Sent: Monday, July 13, 2020 2:04 PM To: National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization Data-Ingestion-and-Harmonization@noreply.github.com Cc: Stephanie Hong shong59@jh.edu; Author author@noreply.github.com Subject: Re: [National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization] No qualifier concept ids defined in the ValueSet Mapping document (#55)
External Email - Use Caution
Hi,
In her review, Kristen suggested using concept_id zero.
Thanks, Charles
From: Stephanie Hong notifications@github.com<mailto:notifications@github.com> Sent: Monday, July 13, 2020 11:54 AM To: National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization Data-Ingestion-and-Harmonization@noreply.github.com<mailto:Data-Ingestion-and-Harmonization@noreply.github.com> Cc: Charles Yaghmour cyaghmour@samvit-solutions.com<mailto:cyaghmour@samvit-solutions.com>; Assign assign@noreply.github.com<mailto:assign@noreply.github.com> Subject: Re: [National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization] No qualifier concept ids defined in the ValueSet Mapping document (#55)
LAB_RESULT_CM RESULT_QUAL ABNORMAL Abnormal See notes Gap LAB_RESULT_CM RESULT_QUAL BORDERLINE Borderline See notes Gap LAB_RESULT_CM RESULT_QUAL ELEVATED Elevated See notes Gap LAB_RESULT_CM RESULT_QUAL HIGH High See notes Gap LAB_RESULT_CM RESULT_QUAL LOW Low See notes Gap LAB_RESULT_CM RESULT_QUAL NEGATIVE Negative See notes Gap LAB_RESULT_CM RESULT_QUAL NI No Information See notes Gap LAB_RESULT_CM RESULT_QUAL NORMAL Normal See notes Gap LAB_RESULT_CM RESULT_QUAL OT Other See notes Gap LAB_RESULT_CM RESULT_QUAL POSITIVE Positive See notes Gap LAB_RESULT_CM RESULT_QUAL UN Unknown See notes Gap LAB_RESULT_CM RESULT_QUAL UNDETECTABLE Undetectable See notes Gap LAB_RESULT_CM RESULT_QUAL UNDETERMINED Undetermined See notes Gap
What concept id should we use for undetectable and undetermined?
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization/issues/55#issuecomment-657641915, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGFG2JR3M6P4TALP7W6Y4J3R3MUZTANCNFSM4OJS4TQQ.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNational-COVID-Cohort-Collaborative%2FData-Ingestion-and-Harmonization%2Fissues%2F55%23issuecomment-657707919&data=02%7C01%7Cshong59%40jh.edu%7Ce47be29cf9f64d078e0808d827571fd1%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637302602434480105&sdata=SOtc9s6bNyeHpedv07v9CU8dd5KDHL4GDBig5t5H4yk%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOVLW5ZXFDLJVSFEMGQL72TR3NEAVANCNFSM4OJS4TQQ&data=02%7C01%7Cshong59%40jh.edu%7Ce47be29cf9f64d078e0808d827571fd1%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637302602434480105&sdata=4OFUoOQVmWC6WNPpEPum%2BNL4BONKRYlXcK7oxCZsnbY%3D&reserved=0.
Hi Charles, In your valueSet mapping document you usually provide all the following column I have listed below. Could you do the same thing for the following list of qualitative codes: I cannot populate the table with missing information.
CDM_TBL
CDM_TBL_COLUMN_NAME
SRC_CODE
SRC_CD_DESCRIPTION
TARGET_CONCEPT_ID
TARGET_CONCEPT_NAME
TARGET_DOMAIN_ID
TARGET_VOCABULARY_ID
TARGET_CONCEPT_CLASS_ID
TARGET_STANDARD_CONCEPT
TARGET_CONCEPT_CODE
LAB_RESULT_CM
RESULT_QUAL
ABNORMAL
Abnormal
45878745
LAB_RESULT_CM
RESULT_QUAL
BORDERLINE
Borderline
45880922
LAB_RESULT_CM
RESULT_QUAL
ELEVATED
Elevated
4328749
LAB_RESULT_CM
RESULT_QUAL
HIGH
High
45876384
LAB_RESULT_CM
RESULT_QUAL
LOW
Low
45881666
LAB_RESULT_CM
RESULT_QUAL
NEGATIVE
Negative
45878583
LAB_RESULT_CM
RESULT_QUAL
NI
No Information
46237210
LAB_RESULT_CM
RESULT_QUAL
NORMAL
Normal
45884153
LAB_RESULT_CM
RESULT_QUAL
OT
Other
46237210
LAB_RESULT_CM
RESULT_QUAL
POSITIVE
Positive
45884084
LAB_RESULT_CM
RESULT_QUAL
UN
Unknown
45877986
LAB_RESULT_CM
RESULT_QUAL
UNDETECTABLE
Undetectable
0
LAB_RESULT_CM
RESULT_QUAL
UNDETERMINED
Undetermined
0
Stephanie From: cyaghmour notifications@github.com Sent: Monday, July 13, 2020 2:04 PM To: National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization Data-Ingestion-and-Harmonization@noreply.github.com Cc: Stephanie Hong shong59@jh.edu; Author author@noreply.github.com Subject: Re: [National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization] No qualifier concept ids defined in the ValueSet Mapping document (#55)
External Email - Use Caution
Hi,
In her review, Kristen suggested using concept_id zero.
Thanks, Charles
From: Stephanie Hong notifications@github.com<mailto:notifications@github.com> Sent: Monday, July 13, 2020 11:54 AM To: National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization Data-Ingestion-and-Harmonization@noreply.github.com<mailto:Data-Ingestion-and-Harmonization@noreply.github.com> Cc: Charles Yaghmour cyaghmour@samvit-solutions.com<mailto:cyaghmour@samvit-solutions.com>; Assign assign@noreply.github.com<mailto:assign@noreply.github.com> Subject: Re: [National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization] No qualifier concept ids defined in the ValueSet Mapping document (#55)
LAB_RESULT_CM RESULT_QUAL ABNORMAL Abnormal See notes Gap LAB_RESULT_CM RESULT_QUAL BORDERLINE Borderline See notes Gap LAB_RESULT_CM RESULT_QUAL ELEVATED Elevated See notes Gap LAB_RESULT_CM RESULT_QUAL HIGH High See notes Gap LAB_RESULT_CM RESULT_QUAL LOW Low See notes Gap LAB_RESULT_CM RESULT_QUAL NEGATIVE Negative See notes Gap LAB_RESULT_CM RESULT_QUAL NI No Information See notes Gap LAB_RESULT_CM RESULT_QUAL NORMAL Normal See notes Gap LAB_RESULT_CM RESULT_QUAL OT Other See notes Gap LAB_RESULT_CM RESULT_QUAL POSITIVE Positive See notes Gap LAB_RESULT_CM RESULT_QUAL UN Unknown See notes Gap LAB_RESULT_CM RESULT_QUAL UNDETECTABLE Undetectable See notes Gap LAB_RESULT_CM RESULT_QUAL UNDETERMINED Undetermined See notes Gap
What concept id should we use for undetectable and undetermined?
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization/issues/55#issuecomment-657641915, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGFG2JR3M6P4TALP7W6Y4J3R3MUZTANCNFSM4OJS4TQQ.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNational-COVID-Cohort-Collaborative%2FData-Ingestion-and-Harmonization%2Fissues%2F55%23issuecomment-657707919&data=02%7C01%7Cshong59%40jh.edu%7Ce47be29cf9f64d078e0808d827571fd1%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637302602434480105&sdata=SOtc9s6bNyeHpedv07v9CU8dd5KDHL4GDBig5t5H4yk%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOVLW5ZXFDLJVSFEMGQL72TR3NEAVANCNFSM4OJS4TQQ&data=02%7C01%7Cshong59%40jh.edu%7Ce47be29cf9f64d078e0808d827571fd1%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637302602434480105&sdata=4OFUoOQVmWC6WNPpEPum%2BNL4BONKRYlXcK7oxCZsnbY%3D&reserved=0.
Hi @stephanieshong ,
I am no longer the custodian of the mapping spreasheet so I can't update it. Please check with Davera. However, I have attached a stand alone file with the information you requested.
Thanks Charles lab_results_cm_results_qual-only.xlsx
@DaveraGabriel @cyaghmour @kmkostka @cukarthik RESULT_QUAL | UNDETECTABLE | Undetectable | 0 |
---|
For Undetectable could we use unknown instead of 0?
@DaveraGabrielhttps://github.com/DaveraGabriel @cyaghmourhttps://github.com/cyaghmour @kmkostkahttps://github.com/kmkostka @cukarthikhttps://github.com/cukarthik RESULT_QUAL
UNDETECTABLE
Undetectable
0
For Undetectable could we use unknown ( 45877393 unknown concept _id ) instead of 0 for concept id? 0 seems that we did not map to anything. Stephanie
From: cyaghmour notifications@github.com Sent: Monday, July 13, 2020 2:59 PM To: National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization Data-Ingestion-and-Harmonization@noreply.github.com Cc: Stephanie Hong shong59@jh.edu; Mention mention@noreply.github.com Subject: Re: [National-COVID-Cohort-Collaborative/Data-Ingestion-and-Harmonization] No qualifier concept ids defined in the ValueSet Mapping document (#55)
External Email - Use Caution
I am no longer the custodian of the mapping spreasheet so I can't update it. Please check with Davera. However, I have attached a stand alone file with the information you requested.
Thanks Charles lab_results_cm_results_qual-only.xlsxhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNational-COVID-Cohort-Collaborative%2FData-Ingestion-and-Harmonization%2Ffiles%2F4914544%2Flab_results_cm_results_qual-only.xlsx&data=02%7C01%7Cshong59%40jh.edu%7C6a4731837a094571fb9908d8275ed0c4%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637302636082475559&sdata=dKPSgZ8t3rPSZidv3GKoL60Btz8vengb5pYveqdGNqs%3D&reserved=0
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNational-COVID-Cohort-Collaborative%2FData-Ingestion-and-Harmonization%2Fissues%2F55%23issuecomment-657734198&data=02%7C01%7Cshong59%40jh.edu%7C6a4731837a094571fb9908d8275ed0c4%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637302636082475559&sdata=OsFcSgnC%2FX3kRwy08Y8ACVSbSPvOnKyVJIDL2bJIh5g%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAOVLW53XAPLMBCQY7IL6ETTR3NKPPANCNFSM4OJS4TQQ&data=02%7C01%7Cshong59%40jh.edu%7C6a4731837a094571fb9908d8275ed0c4%7C9fa4f438b1e6473b803f86f8aedf0dec%7C0%7C0%7C637302636082485553&sdata=XKdLAIYIfBWFJTQFyzLK%2FNVzUzCAECdocTBnOoEW5jw%3D&reserved=0.
'pos' 45884084 'neg' 45878583 'presumptive positive' 45884084 'presumptive negative'45878583 'detected' 45884084 'not detected' 45878583 'inconclusive' 45877990
The mapping process is not case-sensitive and the spaces in the qualitative result fields are trimmed.
@stephanieshong The concept you picked is related to a LOINC code that is supposed to be used as an answer for Race values.
Which is to say: you could map it to this value but there’s a chance most end users would not know to use this VALUE_AS_A_CONCEPT when they were specifying a query. An end user would be looking to pull LOINC codes that are expected to be results for a lab test.
@kmkostka - Can you be more specific? We are not using, 45877393, anywhere.
@stephanieshong You tagged me above and asked: “For Undetectable could we use unknown ( 45877393 unknown concept _id ) instead of 0 for concept id?”
My response is: this code is probably not appropriate.
'un' map to 0 'unknown' map to 0 undetectable should be map to negative 45878583 ? @kmkostka - is this correct?
@stephanieshong Yes for ‘un’ and ‘unknown’. Not sure Undetected is clinically synonymous with Negative. At least in HIV research, it definitely isn’t. I’m not sure what the prevailing clinical wisdom is for COVID. @cgchute or @cukarthik, thoughts?
My apologies for not uploading this work prior to just now. Last week, I took a stab at mapping these nulls that were produced as a result of running the DQD for one site. Had planned to discus in our mapping validation meeting scheduled for tomorrow. The urgency of the N3C soft launch has preempted this delay.
fkDomain Check Failure Concept Codes_DG_Comments6JUL2020.xlsx
@stephanieshong @kmkostka
Other, No Information, and Unknowns are now mapped to 0 . (OT, NI, and UN are mapped to 0)
Could not find any valueSet mapping for the qualifier concept. Please validate if this is correct: case when lower(trim(result_qual)) = 'positive' then 45884084 when lower(trim(result_qual)) = 'negative' then 45878583 when lower(trim(result_qual)) = 'normal' then 45884153 when lower(trim(result_qual)) = 'abnormal' then 45878745 when lower(trim(result_qual)) = 'low' then 45881666 when lower(trim(result_qual)) = 'high' then 45876384 when lower(trim(result_qual)) = 'borderline' then 45880922 when lower(trim(result_qual)) = 'elevatied' then 3445772 --ssh add issue number when lower(trim(result_qual)) = 'un' then 45880480 else 45877393 end as QUALIFIER_CONCEPT_ID --null/un/elevated/boderline/ot/low/high/normal/negative/positive/abnormal