search

OHDSI / OMOPV4_PCORNetV1_ETL

ETL script to transform data from OMOP v4 CDM to PCORNet V1 CDM
9 stars 7 forks source link

What are the advantages of using OMOP Vocabulary V4.5? #7

Closed gracebrownecodes closed 9 years ago

gracebrownecodes commented 9 years ago

Making an issue to hold discussion of moving PEDSnet to OMOP Vocab 4.5 in order to support the transformation to PCORnet V1. According to a recent email chain, this is a larger change than I thought. Can someone explain to me two things:

  1. What parts of Vocab 4.5 are required for transformation?
  2. What are the differences between V4 and V4.5 (I only have the V4.5 data itself, not DDL or documentation)?
cgreich commented 9 years ago

Aaron:

See below:

From: Aaron Browne [mailto:notifications@github.com] Sent: Monday, October 13, 2014 10:19 AM To: OHDSI/OMOPV4_PCORNetV1_ETL Subject: [OMOPV4_PCORNetV1_ETL] What are the advantages of using OMOP Vocabulary V4.5? (#7)

Making an issue to hold discussion of moving PEDSnet to OMOP Vocab 4.5 in order to support the transformation to PCORnet V1. According to a recent email chain, this is a larger change than I thought. Can someone explain to me two things:

  1. What parts of Vocab 4.5 are required for transformation?

Don't understand the question. All of them. Or what do you mean?

  1. What are the differences between V4 and V4.5 (I only have the V4.5 data itself, not DDL or documentation)?

There is no V4. There is the previous V4.4, but the differences are incremental. Look at the release notes. I don't think it is very relevant to what you are doing at all, except maybe the consolidated mapping_type records in the source_to_cocnept_map table.

C

— Reply to this email directly or view it on GitHub https://github.com/OHDSI/OMOPV4_PCORNetV1_ETL/issues/7 .Image removed by sender.

gracebrownecodes commented 9 years ago

Sorry @cgreich, this was aimed more at @toanong and @mgkahn.

As I understand it, there is some part of the OMOP->PCORnet transformation that will work or work better with Vocab V4.5, and I'm trying to figure out exactly what that is.

cgreich commented 9 years ago

Shouldn't make a difference. If anything, v5.0 should be easier, except nobody has it yet. :-)

From: Aaron Browne [mailto:notifications@github.com] Sent: Monday, October 13, 2014 12:16 PM To: OHDSI/OMOPV4_PCORNetV1_ETL Cc: Christian Reich Subject: Re: [OMOPV4_PCORNetV1_ETL] What are the advantages of using OMOP Vocabulary V4.5? (#7)

Sorry @cgreich https://github.com/cgreich , this was aimed more at @toanong https://github.com/toanong and @mgkahn https://github.com/mgkahn .

As I understand it, there is some part of the OMOP->PCORnet transformation that will work or work better with Vocab V4.5, and I'm trying to figure out exactly what that is.

— Reply to this email directly or view it on GitHub https://github.com/OHDSI/OMOPV4_PCORNetV1_ETL/issues/7#issuecomment-58915835 .Image removed by sender.

gracebrownecodes commented 9 years ago

Some things that I have heard as reasoning is that somewhere in between the version we have now and 4.5 more condition source_to_concept mappings were added (which would increase our capture of OMOP conditions and the corresponding generation of PCORnet diagnoses) and also that "flavors of null" have been added to allow the PCORnet-expected "No Information/Unknown/Other" distinction.

mgkahn commented 9 years ago

We have one and only one reason for why we want Vocabulary 4.5: Between Vocabulary 4.3 and 4.5, additional information was added to DRG concepts in the CONCEPT table (vocabulary_id = 40). Specifically, valid_start_date and valid_end_date were changed to represent correct dates for when CMS switched from CMS-DRGs to MS-DRGs (10/1/2007) and also the invalid_reason field was changed from all NULLs in Version 4.3 to reasonable reason codes. Toan's SQL for converting source DRGs into correct CONCEPT_IDs and then into correct PCORnet values uses these changes to find the right CONCEPT_IDs especially if the data crosses the 10/1/2007 date boundary when CMS switched to MS-DRGs.

Hence, our _ONLY_ reason for using Vocabulary 4.5 is just because we need these improvements in Vocabulary_ID = 40 in the CONCEPT table.

Does this make sense?

cgreich commented 9 years ago

Michael!!!! We worked our behinds off to make dozens of improvements, and the only value you see is the DRG start_dates? :-):-)

See you tomorrow.

C

From: Michael Kahn [mailto:notifications@github.com] Sent: Monday, October 13, 2014 10:07 PM To: OHDSI/OMOPV4_PCORNetV1_ETL Cc: Christian Reich Subject: Re: [OMOPV4_PCORNetV1_ETL] What are the advantages of using OMOP Vocabulary V4.5? (#7)

We have one and only one reason for why we want Vocabulary 4.5: Between Vocabulary 4.3 and 4.5, additional information was added to DRG concepts in the CONCEPT table (vocabulary_id = 40). Specifically, valid_start_date and valid_end_date were changed to represent correct dates for when CMS switched from CMS-DRGs to MS-DRGs (10/1/2007) and also the invalid_reason field was changed from all NULLs in Version 4.3 to reasonable reason codes. Toan's SQL for converting source DRGs into correct CONCEPT_IDs and then into correct PCORnet values uses these changes to find the right CONCEPT_IDs especially if the data crosses the 10/1/2007 date boundary when CMS switched to MS-DRGs.

Hence, our ONLY reason for using Vocabulary 4.5 is just because we need these improvements in Vocabulary_ID = 40 in the CONCEPT table.

Does this make sense?

— Reply to this email directly or view it on GitHub https://github.com/OHDSI/OMOPV4_PCORNetV1_ETL/issues/7#issuecomment-58980944 .Image removed by sender.

gracebrownecodes commented 9 years ago

@cgreich Are the concept_ids consistent with past versions? If we just load the (or some) concept records from the new vocab version, will it work with tables from the previous version?

pbr6cornell commented 9 years ago

yes, concept_ids do not get re-used, and are consistent across versions. note, some concepts in past versions may become deprecated by SNOMED or RxNorm, so would be recommended to no longer be used in future CDM ETL runs.

On Wed, Oct 15, 2014 at 2:25 PM, Aaron Browne notifications@github.com wrote:

@cgreich https://github.com/cgreich Are the concept_ids consistent with past versions? If we just load the (or some) concept records from the new vocab version, will it work with tables from the previous version?

— Reply to this email directly or view it on GitHub https://github.com/OHDSI/OMOPV4_PCORNetV1_ETL/issues/7#issuecomment-59251945 .