FINNGEN / ETL

second version for the ETL of the detailed longitudinal data to the OMOP-CDM
https://finngen.github.io/ETL/
MIT License
0 stars 0 forks source link

ETL birth mother registry is done. All tests PASS. #127

Closed shanmukhasampath closed 1 year ago

shanmukhasampath commented 1 year ago

This is for issue #120

Birth Mother registry ETL is finished.

  1. Added necessary setup functions for birth mother registry without disturbing other registries
  2. Changed TestFramework.R to be able to do unittests.
  3. SQL scripts to run ETL of birth mother registry data
  4. Test scripts in R for unit testing
  5. Changed input DEATH registry to death_registry which makes the unit tests work properly
javier-gracia-tabuenca-tuni commented 1 year ago

id like to see config/run_config_template.R

javier-gracia-tabuenca-tuni commented 1 year ago

from slack

what if we put all the unittest_etl_birthmother* files into one file unittest_etl_birth_mother_event_tables.R ?? 2:31 and we have it in "birth_mother stem_medical_events" , here::here("sql/etl_birth_mother_stem_medical_events.sql"), "unittest_etl_birth_mother_event_tables.R", TRUE, TRUE,

shanmukhasampath commented 1 year ago

Combined birth mother unit tests for condition, procedure, observation and measurement tables into one file.

Should the run_config_template.R also be updated?

javier-gracia-tabuenca-tuni commented 1 year ago

add an offset to the visit_occurrence_id, to make it independent of the size of other registers for example 1 000 000 000 000 + mother visit I guess in finregister is not gonnabe more than 1B visits ?

shanmukhasampath commented 1 year ago

Fixed the visit_occurrence table for birth mother with offset.

shanmukhasampath commented 1 year ago

Birth mother visit and stem medical events are ready to be published. Initial versions takes care of the following scenarios

Improvements will be followed in another issue.