Update vocabulary tables

[DrUrwin]

Update vocabulary tables to include all SNOMED codes.

Issue:

Cannot map metadata items to 6 digit concept IDs, e.g.:

607590 | 1153637007 | Body height

CARROT refuses to accept the mapping for a 'Valid & Standard' vocabulary concept ID.

[DrUrwin]

Additionally, cannot map to:

903654 | OMOP5181835 | Tobacco or its derivatives user

[erummas]

Hi @DrUrwin, it it most probably due to the fact that the vocabulary was never updated after 2020. These are new concepts and currently unavailable on the mapper. Once the vocabs are updated, it should resolve. #624

[AndyRae]

The vocabularies have now been updated to the latest downloaded on 28/03/2024. If there are problems, we can quickly rollback and restore the archived CDM.

The following vocabularies were removed:

• ICD10_UK
• ICD9CM_UK
• OPCS4_UK

And added:

• NAACCR

For future, here's the current update process:

• [ ] 1. Check the vocabulary table of the CDM you want to update.
• [ ] 2. Select these on Athena to download.
• [ ] 3. Download them
• [ ] 4. (do any transformation on them that is necessary, e.g dates)
• [ ] 5. upload them to Azure storage.
• [ ] 6. Backup existing omop schema.
• [ ] 7. Run Azure Data Factory to insert vocabs into staging schema.
• [ ] 8. Run SQL to add to staging:
  • [ ] primary keys
  • [ ] constraints,
  • [ ] indexes
• [ ] 9. Create archive schema omop_{date}
• [ ] 10. Run SQL to set omop tables to the archive schema
• [ ] 11. Run SQL to set staging tables to the omop schema