search

Medical-Event-Data-Standard / meds_etl

A collection of ETLs from common data formats to Medical Event Data Standard
Apache License 2.0
16 stars 3 forks source link

Add testing framework for MIMIC-IV ETL #1

Closed tompollard closed 5 months ago

tompollard commented 6 months ago

We could consider adding tests for these ETLs, e.g. the MIMIC-IV ETL should be reproducible using the open MIMIC-IV demo at: https://doi.org/10.13026/dp1f-ex47.

If this sounds like a good idea, we could either (1) include a copy of the data in this repo (2) make the tests dependent on an externally hosted version of the data.

My preference is 2 I think, just to help keep this repo clean.

The MIMIC-IV demo doesn't appear to be on BigQuery at the moment, but we can arrange this. We could then set up GitHub actions to run the tests against BigQuery.

EthanSteinberg commented 6 months ago

Yep, this is a really good idea!

We can use the MIMIC demo for MIMIC and a synthetic OMOP dataset for OMOP. I already have a synthetic OMOP on hand from https://github.com/som-shahlab/femr/tree/main/tutorials/input/omop which will probably be sufficient for that.

EthanSteinberg commented 6 months ago

https://github.com/Medical-Event-Data-Standard/meds_etl/actions seems to now be working!

tompollard commented 6 months ago

Perfect, thanks Ethan! I'll have a go at adding a couple of MIMIC tests later today.