search

google / fhir-data-pipes

A collection of tools for extracting FHIR resources and analytics services on top of that data.
https://google.github.io/fhir-data-pipes/
Apache License 2.0
153 stars 86 forks source link

Use Synthea to generate HIV datasets and import to OpenMRS #179

Closed bashir2 closed 2 years ago

bashir2 commented 3 years ago

This is our second attempt to generate synthetic data (previous one is #59 which is stalled). The idea here is to use Synthea and create/reuse/extend one of its modules for HIV and maybe mix with TB too. There seems to be an experimental HIV module here which we can probably adopt.

The generated FHIR resources should be imported to OpenMRS and the end result is a dump of the MySQL DB which we can use for our tests (instead of current demo-data we use). We probably need datasets of different sizes, e.g., 10s, thousands, and 100s of thousands of patients to fit both quick development scenarios and performance intensive ones.

Some Synthea resources:

kimaina commented 3 years ago

Also here is a CSV containing concept dictionary distribution from AMPATH (over 90% of the concept ids are based on CIEL). concept dictionary distribution .csv

kimaina commented 3 years ago

Hi @bashir2 and @omarismail94 . Following the discussion on #196 I am formally documenting the list of essential codes (Q&A), that are frequently used @AMPATH and needed for generating the 10 PEPFAR indicators.

Question code Answer Codes Sytem Mappings
1. HIV RAPID TEST, QUALITATIVE (1040), NEGATIVE (664), POSITIVE (703), POOR SAMPLE QUALITY (1304), INDETERMINATE (1138), UNKNOWN (1067), NOT DONE (1118), INVALID (10766), CIEL, AMPATH
1. ANTIRETROVIRAL PLAN (1255), CONTINUE REGIMEN (1257), NONE (1107), START DRUGS (1256), STOP ALL MEDICATIONS (1260), CHANGE REGIMEN (1259), DOSING CHANGE (981), CHANGE FORMULATION (1258), NOT REFILLED (1407), REFILLED (1406), DRUG SUBSTITUTION (1849), DRUG RESTART (1850), CIEL, AMPATH
1. CURRENT ANTIRETROVIRAL DRUGS USED FOR TREATMENT (1088), 2. PATIENT REPORTED CURRENT ANTIRETROVIRAL TREATMENT(2154), 3. ANTIRETROVIRALS STARTED (1250), NELFINAVIR (635), LOPINAVIR 80mg AND RITONAVIR 20mg SOL (LPV 80mg AND RIT 20mg SOL), (LOPINAVIR AND RITONAVIR), (794), OTHER ANTIRETROVIRAL DRUG (5424), UNKNOWN ANTIRETROVIRAL DRUG (5811), RITONAVIR 100mg CAP(NORVIR SEC 100mg CAP), (RITONAVIR), (795), DARUNAVIR (6157), RITONAVIR 100mg TAB(NORVIR 100mg TAB), (RITONAVIR), (795), ATAZANAVIR (6159), ETRAVIRINE (6158), RITONAVIR (795), ABACAVIR LAMIVUDINE AND ZIDOVUDINE (817), EMTRICITABINE (791), NONE (1107), RILPIVIRINE (10090), EVIPLERA (9435), DOLUTEGRAVIR (9759), ABACAVIR LAMIVUDINE LOPINAVIR AND RITONAVIR (9051), RITONAVIR 80mg/ml SYR (NORVIR 80mg/ml SYR), (RITONAVIR), (795), EMTRICITABINE 200mg AND RIPLIVIRINE 25mg AND TENOFOVIR 245mg(FTC 200mg AND RPV 25mg AND TDF 245mg), (EVIPLERA), (9435), DOLUTEGRAVIR 50mg (DOLUTEGRAVIR), (9759), EFAVIRENZ AND LAMIVUDINE AND TENOFOVIR DISOPROXIL (10795), DOLUTEGRAVIR 50mg AND LAMIVUDINE 300mg AND TENOFOVIR 300mg (DOLUTEGRAVIR LAMIVUDINE AND TENOFOVIR), (10268), ATAZANAVIR AND RITONAVIR (6160), LOPINAVIR AND RITONAVIR (794), LAMIVUDINE AND TENOFOVIR (1400), DDI 200 (DIDANOSINE), (796), NEVIRAPINE (631), LAMIVUDINE (628), ZIDOVUDINE (797), d4T-40 (STAVUDINE), (625), ABACAVIR (814), DOLUTEGRAVIR LAMIVUDINE AND TENOFOVIR (10268), LOPINAVIR 40mg AND RITONAVIR 10mg (LOPINAVIR AND RITONAVIR), (794), TENOFOVIR (802), TENOFOVIR 300mg AND LAMIVUDINE 300mg TAB (TDF 300mg AND 3TC 300mg TAB), (LAMIVUDINE AND TENOFOVIR), (1400), EMTRICITABINE AND TENOFOVIR (6180), INDINAVIR (749), NEVIRAPINE LAMIVUDINE AND ZIDOVUDINE (6467), Triomune-40 (STAVUDINE LAMIVUDINE AND NEVIRAPINE), (792), EFAVIRENZ (633), 3TC 150 and D4T 30 (LAMIVUDINE AND STAVUDINE), (6965), TDF 300 AND 3TC 300 AND EFV 600 (TENOFOVIR AND LAMIVUDINE AND EFAVIRENZ), (6964), RALTEGRAVIR (6156), ABACAVIR AND LAMIVUDINE (6679), ZIDOVUDINE AND LAMIVUDINE (630), DIDANOSINE (796), d4T-30 (STAVUDINE), (625), Triomune-30 (STAVUDINE LAMIVUDINE AND NEVIRAPINE), (792), DDI 125 (DIDANOSINE), (796), STAVUDINE (625), STAVUDINE LAMIVUDINE AND NEVIRAPINE (792), ATAZANAVIR 300mg AND RITONAVIR 100mg TAB (ATAZANAVIR AND RITONAVIR), (6160), Triomune-15 (STAVUDINE LAMIVUDINE AND NEVIRAPINE), (792), Triomune-20 (STAVUDINE LAMIVUDINE AND NEVIRAPINE), (792), DDI 400 EC (DIDANOSINE), (796), DDI 250 EC (DIDANOSINE), (796), Triomune-Junior (STAVUDINE LAMIVUDINE AND NEVIRAPINE), (792), 3TC 30 and NVP 50 and ZDV 60 (NEVIRAPINE LAMIVUDINE AND ZIDOVUDINE), (6467), NVP 200 and 3TC 150 and ZDV 300 (NEVIRAPINE LAMIVUDINE AND ZIDOVUDINE), (6467), NEVIRAPINE 200mg (NEVIRAPINE), (631), LAMIVUDINE 150mg (LAMIVUDINE), (628), ZIDOVUDINE 300mg (ZIDOVUDINE), (797), STAVUDINE 30mg (STAVUDINE), (625), ABACAVIR 300mg (ABACAVIR), (814), LPV 200 and RIT 50 (LOPINAVIR AND RITONAVIR), (794), TENOFOVIR 300mg (TENOFOVIR), (802), 3TC 300 and TDF 300 (LAMIVUDINE AND TENOFOVIR), (1400), EMTRICITABINE 200 and TENOFOVIR 300 (EMTRICITABINE AND TENOFOVIR), (6180), INDINAVIR 400mg (INDINAVIR), (749), NVP and 3TC and AZT (NEVIRAPINE LAMIVUDINE AND ZIDOVUDINE), (6467), NVP 200 and D4T 30 and 3TC 150 (STAVUDINE LAMIVUDINE AND NEVIRAPINE), (792), EFAVIRENZ 600mg (EFAVIRENZ), (633), 3TC 150 and ZDV 300 (ZIDOVUDINE AND LAMIVUDINE), (630), LAMIVUDINE AND STAVUDINE (6965), TENOFOVIR AND LAMIVUDINE AND EFAVIRENZ (6964), RALTEGRAVIR 400mg (RALTEGRAVIR), (6156), ABC 60 and 3TC 30 (ABACAVIR AND LAMIVUDINE), (6679), 3TC 30 and ZDV 60 (COMBIVIR JR), (ZIDOVUDINE AND LAMIVUDINE), (630), CIEL, AMPATH
1. REVIEW OF TUBERCULOSIS SCREENING QUESTIONS (6174), HOUSEHOLD MEMBER DIAGNOSED WITH TUBERCULOSIS (2020), FEVER MORE THAN THREE WEEKS (6173), BLOODY COUGH IN PAST YEAR (6172), COUGH FOR MORE THAN TWO WEEKS (6171), FAILURE TO THRIVE (5050), WEIGHT LOSS (832), NECK SWELLING (8059), AXILLARY SWELLING (8060), ABDOMINAL SWELLING (6019), JOINT SWELLING (5312), NIGHT SWEATS MORE THAN TWO WEEKS (8061), FEVER MORE THAN TWO WEEKS (8065), GROIN SWELLING (8066), NEW EXPOSURE TO HOUSEHOLD CONTACT WITH TUBERCULOSIS (8067), NONE (1107), CHEST PAIN (136), SHORTNESS OF BREATH (5960), FEVER (5945), BACK PAIN (456), OTHER NON-CODED (5622), COUGH (107), CIEL, AMPATH
1. TUBERCULOSIS DIAGNOSED THIS VISIT (2022), NO (1066), YES (1065), TUBERCULOSIS TREATMENT DRUGS (1159), CIEL, AMPATH
1. TUBERCULOSIS TREATMENT PLAN (1268), CONTINUE REGIMEN (1257), NONE (1107), START DRUGS (1256), STOP ALL MEDICATIONS (1260), CHANGE REGIMEN (1259), DOSING CHANGE (981), DRUG RESTART (1850), TUBERCULOSIS DEFAULTER REGIMEN BY USING STREPTOMYCIN (2160), MULTIDRUG-RESISTANT TUBERCULOSIS REGIMEN (2161), REFILLED (1406), NOT REFILLED (1407), DRUG SUBSTITUTION (1849), CONTINUE TO RECEIVE MEDICATIONS FROM ANOTHER LOCATION (8352), CIEL, AMPATH
1. TUBERCULOSIS TREATMENT STARTED (1270), STREPTOMYCIN (438), RIFAMPICIN AND ISONIAZID (1194), RIFAMPICIN (767), OTHER NON-CODED (5622), PEDIATRIC RIFAMPICIN AND ISONIAZID (6603), PEDIATRIC RIFAMPICIN ISONIAZID AND PYRAZINAMIDE (6602), RIFAMPICIN 150mg ISONIAZID 75mg AND PYRAZINAMIDE 400mg TAB (RHZ), (RIFAMPICIN ISONIAZID AND PYRAZINAMIDE), (768), RIFAMPICIN ISONIAZID AND PYRAZINAMIDE (768), RIFABUTIN 150mg TAB (RFB), (RIFABUTIN), (6983), RIFABUTIN (6983), ISONIAZID 300mg TAB (ISONIAZID), (656), ISONIAZID (656), PYRAZINAMIDE 500mg (Z), (PYRAZINAMIDE), (5829), PYRAZINAMIDE (5829), ETHAMBUTOL 400mg AND ISONIZAID 150mg TAB (EH), (ETHAMBUTOL AND ISONIZAID), (1108), ETHAMBUTOL AND ISONIZAID (1108), ETHAMBUTOL 400mg (E), (ETHAMBUTOL), (745), ETHAMBUTOL (745), RIFAMPICIN 150mg ISONIAZID 75mg AND ETHAMBUTOL 275mg TAB (RHE), (RIFAMPICIN ISONIAZID AND ETHAMBUTOL), (2231), RIFAMPICIN ISONIAZID AND ETHAMBUTOL (2231), MULTIDRUG-RESISTANT TUBERCULOSIS REGIMEN (2161), RIFAMPICIN 150mg ISONIAZID 75mg PYRAZINAMIDE 400mg AND ETHAMBUTOL 275mg TAB (RHZE), (RIFAMPICIN ISONIAZID PYRAZINAMIDE AND ETHAMBUTOL), (1131), RIFAMPICIN ISONIAZID PYRAZINAMIDE AND ETHAMBUTOL (1131), CIEL, AMPATH
2. PATIENT REPORTED CURRENT TB TREATMENT (1111), PYRAZINAMIDE (5829), STREPTOMYCIN (438), RIFAMPICIN ISONIAZID PYRAZINAMIDE AND ETHAMBUTOL (1131), NONE (1107), ETHAMBUTOL (745), RIFAMPICIN ISONIAZID AND PYRAZINAMIDE (768), ISONIAZID (656), RIFAMPICIN AND ISONIAZID (1194), OTHER NON-CODED (5622), RIFAMPICIN (767), ETHAMBUTOL AND ISONIZAID (1108), COMPLETED (1267), RIFAMPICIN ISONIAZID AND ETHAMBUTOL (2231), YES (1065), PEDIATRIC RIFAMPICIN ISONIAZID AND PYRAZINAMIDE (6602), PEDIATRIC RIFAMPICIN AND ISONIAZID (6603), RIFABUTIN (6983), MULTIDRUG-RESISTANT TUBERCULOSIS REGIMEN (2161), PATIENT DEFAULTED (1595), AMIKACIN (11717), CAPREOMYCIN SULFATE (7917), KANAMYCIN SULFATE (7860), CIPROFLOXACIN (740), OFLOXACIN (7920), LEVOFLOXACIN (755), MOXIFLOXACIN (955), ETHIONAMIDE (11718), CYCLOSERINE (7677), AMINOSALICYLIC ACID (7918), THIOACETAZONE (11719), AMOXICILLIN AND CLAVULANIC ACID (450), CLARYTHROMYCIN (8922), CLOFAZIMINE (11720), TERIZIDONE (11721), VIOMYCIN (11722), PYRIDOXINE (766), GATIFLOXACIN (11723), CIEL, AMPATH
DATE OF DEATH (1570), Date CIEL, AMPATH
CAUSE FOR DEATH (573), HEALTH ISSUES (1548), ACCIDENT, NOT OTHERWISE SPECIFIED (84), OTHER NON-CODED (5622), MURDER (1572), SUICIDE (1571), PATIENT DIED (1593), MATERNAL DEATH (7257), UNKNOWN (1067), DIARRHEA (16), ACCIDENT, MOTOR VEHICLE (86), KAPOSI'S SARCOMA (507), HIV STAGING - LYMPHOMA (5041), MALARIA (123), MENINGITIS, NOS (60), PNEUMONIA (43), TUBERCULOSIS (58), TOXICITY, DRUG (102), INJURY (148), HIV RESULTING IN CANCER DISEASE (10363), HIV RESULTING IN INFECTIOUS OR PARASITIC DISEASE (10364), NON INFECTIOUS HIV RELATED DISEASE (10365), NATURAL CAUSE (10366), DEATH DUE TO NON NATURAL CAUSE (10367), CIEL, AMPATH
1. HIV VIRAL LOAD, QUANTITATIVE (856), mean: 10,000 and IQR: 0 to 1000000 CIEL, AMPATH
1. TESTS ORDERED (1271), COMPLETE BLOOD COUNT (1019), HEMOGLOBIN (21), SERUM GLUTAMIC-PYRUVIC TRANSAMINASE (654), CD4 PANEL (657), HIV VIRAL LOAD, QUANTITATIVE (856), HIV ENZYME IMMUNOASSAY, QUALITATIVE (1042), HIV DNA POLYMERASE CHAIN REACTION, QUALITATIVE (1030), SERUM CREATININE (790), OTHER NON-CODED (5622), SERUM GLUTAMIC-OXALOACETIC TRANSAMINASE (653), SERUM ELECTROLYTES (5473), TUBERCULIN SKIN TEST, QUANTITATIVE (5475), SPUTUM FOR ACID FAST BACILLI (307), URINE PREGNANCY TEST, QUALITATIVE (45), NONE (1107), SYPHILIS - VDRL, QUALITATIVE (299), URINALYSIS (302), BRUCELLA TEST (305), X-RAY, SKULL (386), X-RAY, ABDOMEN (101), X-RAY, SHOULDER (394), BLOOD TYPING (300), X-RAY, HAND (382), SERUM GLUCOSE (887), X-RAY, ARM (377), WIDAL TEST (306), X-RAY, FOOT (384), X-RAY, PELVIS (392), PAPANICOLAOU SMEAR (885), X-RAY, SPINE (390), X-RAY, LEG (380), MALARIAL SMEAR (32), HERPES SIMPLEX VIRUS, QUALITATIVE (908), HIV RAPID TEST, QUALITATIVE (1040), HERPES SIMPLEX VIRUS, DNA POLYMERASE CHAIN REACTION, QUALITATIVE (1046), TRICHOMONAS VAGINALIS, DNA POLYMERASE CHAIN REACTION, QUALITATIVE (1039), HEPATITIS C TEST, QUALITATIVE (1325), CHLAMYDIA TRACHOMATIS, DNA POLYMERASE CHAIN REACTION, QUALITATIVE (1034), HIV WESTERN BLOT, BANDS, QUANTITATIVE (1045), CHEMISTRY LAB TESTS (1353), HEPATITIS B TEST, QUALITATIVE (1322), NEISSERIA GONORRHOEAE, DNA POLYMERASE CHAIN REACTION, QUALITATIVE (1038), HIV VIRAL LOAD, QUALITATIVE (1305), FINE NEEDLE ASPIRATION, LUNG (1923), ECHOCARDIOGRAPHY (1536), BARIUM MEAL (1512), STOOL EXAM (304), ULTRASOUND, ABDOMEN (845), ELECTROCARDIOGRAPHY (1537), BARIUM SWALLOW (1513), COMPUTED TOMOGRAPHY SCAN, HEAD (846), CULTURE FOR ACID FAST BACILLI (2311), LIVER FUNCTION TESTS (953), MEDICAL IMAGING (6500), CT SCAN (6501), ULTRASOUND (6502), CD4, BY FACS (5497), CD4%, BY FACS (730), ANTISTREPTOLYSIN O TITRE SERUM (6832), XRAY (6897), RENAL FUNCTION BLOOD TEST (6898), PERIPHERAL BLOOD FILM (6901), BONE MARROW ASPIRATION (6902), GLYCOSYLATED HEMOGLOBIN (6126), MICROALBUMIN TEST, URINE (7302), SERUM POTASSIUM (1133), SERUM GLUCOSE, FASTING (6252), SERUM BLOOD UREA NITROGEN (857), LIPID PANEL (1010), FULL THYROID PANEL (7872), FASTING LIPID PANEL (7658), GENEXPERT (8064), ERYTHROCYTE SEDIMENTATION RATE, MICROSCOPIC EXAM (855), RADIOLOGY (6572), CHEST XRAY, PRELIMINARY FINDINGS (12), HIV ULTRA RAPID TEST, QUALITATIVE (8708), SERUM PROTEIN ELECTROPHORESIS (8595), URINE PROTEIN ELECTROPHORESIS, 24 HOURS (8596), X-RAY, OTHER (309), CARDIAC ENZYME TEST (8991), HEMOGLOBIN F (9010), HEMOGLOBIN A (9011), HEMOGLOBIN A2 (9012), HIGH VAGINAL SWAB (1987), UROGENITAL (1005), MAGNETIC RESONANCE IMAGING (9550), HBV ANTIGEN (1375), CRYPTOCOCCAL ANTIGEN TEST (9812), COMPUTED TOMOGRAPHY SCAN, NECK (9839), COMPUTED TOMOGRAPHY SCAN, CHEST (7113), COMPUTED TOMOGRAPHY SCAN, ABDOMINAL (7114), COMPUTED TOMOGRAPHY SCAN, SPINE (9840), MAGNETIC RESONANCE IMAGING, HEAD (9881), MAGNETIC RESONANCE IMAGING, NECK (9882), MAGNETIC RESONANCE IMAGING, CHEST (9883), MAGNETIC RESONANCE IMAGING, ARMS (9951), MAGNETIC RESONANCE IMAGING, ABDOMEN (9884), MAGNETIC RESONANCE IMAGING, PELVIC (9952), MAGNETIC RESONANCE IMAGING, SPINE (9885), MAGNETIC RESONANCE IMAGING, LEGS (9953), ULTRASOUND, RENAL (7115), ULTRASOUND, HEPATIC (852), OBSTETRIC ULTRASOUND (6221), ULTRASOUND, BREAST (9596), RADIOLOGY TEST, FREETEXT (8190), DRUG SENSITIVITY TEST (9920), PET SCAN (10123), HEMOGLOBIN WITH ELECTROPHORESIS (9009), RETICULOCYTES PERCENTAGE, MICROSCOPIC EXAM (1327), SERUM FREE LIGHT CHAIN TEST (10205), IMMUNOHISTOCHEMISTRY (10125), ANTIBODY SCREEN (6342), TUBERCULOSIS POLYMERASE CHAIN REACTION WITH RIFAMPIN RESISTANCE CHECKING (10761), URINALYSIS ORDER, CODED (11676), CIEL, AMPATH
1. HIV DNA POLYMERASE CHAIN REACTION, QUALITATIVE (1030), NDETERMINATE (1138), NEGATIVE (664), POSITIVE (703), POOR SAMPLE QUALITY (1304), NOT DONE (1118), CIEL, AMPATH
1. DISCONTINUE FROM CLINIC, HIV NEGATIVE (1946), NO (1066), DO NOT KNOW (1624), YES (1065), CIEL, AMPATH
1.TRANSFER CARE TO OTHER CENTER (1285), AMPATH (1286), NON-AMPATH (1287), AMPATH, NON-AMRS (9068), MATERNAL CHILD HEALTH PROGRAM (2050), CIEL, AMPATH
VITALS: # 5089 = WEIGHT # 5090 = HEIGHT (CM), # 5088 = TEMPERATURE (C), 5092 = BLOOD OXYGEN SATURATION # 5085 = SYSTOLIC BLOOD PRESSURE # 5086 = DIASTOLIC BLOOD PRESSURE # 5087 = PULSE # 980 = BSA # 6584 = ECO Quantitative CIEL, AMPATH
CURRENT VISIT TYPE (1839), UNSCHEDULED VISIT LATE (1838), UNSCHEDULED VISIT EARLY (1837), SCHEDULED VISIT (1246), FIRST DISPENSARY VISIT AFTER HCT (7034), SECOND DISPENSARY VISIT AFTER HCT (7035), RETURN DISPENSARY VISIT (7036), REFERRED FROM CLINIC (7037), COMMUNITY HEALTH WORKER INITIAL VISIT (7250), COMMUNITY HEALTH WORKER BIANNUAL REVISIT (7251), COMMUNITY HEALTH WORKER NEW EVENT (7252), COMMUNITY HEALTH WORKER FOLLOW-UP VISIT (7247), FOLLOW-UP (2345), REVIEW OF PATHOLOGY RESULTS (7588), INITIAL VISIT (7850), READMISSION (7930), RELAPSE (7931), HOME VISIT (7066), ORPHANS AND VULNERABLE CHILDREN CLINICAL ENCOUNTER (8615), OTHER NON-CODED (5622), UNSCHEDULED OUTPATIENT VISIT (8914), END OF WARFARIN THERAPY (8915), FIRST DISPENSARY VISIT AFTER SCREENING (9456), SECOND DISPENSARY VISIT AFTER SCREENING (9457), WALK IN (7875), PEER ADMISSION (9570), PEER DISCHARGE (9571), PEER FOLLOW-UP VISIT (9569), START INSURANCE (10305), CHANGE INSURANCE (10306), REFERRED FROM CLINICAL OFFICER (10371), REFERRED FROM GROUP CARE (10372), REFERRED FROM OUT PATIENT CLINIC (10374), INPATIENT PEER VISIT (10383), MEDICATION REFILL BETWEEN VISIT (10384), TREATMENT SUPPORTER VISIT (10385), DIFFERENTIATED CARE COMMUNITY VISIT (10386), DIFFERENTIATED CARE PHARMACY VISIT (10387), REFERRED FROM SCREENING (11058), RETURN VISIT TYPE (11233), CIEL, AMPATH
1.TUBERCULOSIS PROPHYLAXIS PLAN (1265), 2. PCP PROPHYLAXIS PLAN (1261), CONTINUE REGIMEN (1257), NONE (1107), START DRUGS (1256), STOP ALL MEDICATIONS (1260), DOSING CHANGE (981), NOT REFILLED (1407), REFILLED (1406), DRUG RESTART (1850), DRUG SUBSTITUTION (1849), CIEL, AMPATH
RETURN VISIT DATE (5096), Date CIEL, AMPATH
2. PRE-EXPOSURE PROPHYLAXIS PLAN (9772), CONTINUE REGIMEN (1257), DRUG RESTART (1850), DISCONTINUE (6102), CIEL, AMPATH
CIEL, AMPATH
kimaina commented 3 years ago

Also this summary stats on frequency of clinical encounters can be useful: https://github.com/GoogleCloudPlatform/openmrs-fhir-analytics/issues/59#issuecomment-732715292

omarismail94 commented 3 years ago

Thanks @kimaina ! Starting to take a look at all these resources today

omarismail94 commented 3 years ago

What I've done so far:

How to generate data:

Upload the data to a local or GCP FHIR Server.

The commands below need to be done in order.

For the local server:

for FILE in output/fhir/hospital*.json; do curl -X POST -H "Content-Type: application/fhir+json;charset=utf-8" --data-binary @$FILE http://localhost:8098/fhir ; done

for FILE in output/fhir/practitioner*.json; do curl -X POST -H "Content-Type: application/fhir+json;charset=utf-8" --data-binary @$FILE http://localhost:8098/fhir ; done

for FILE in output/fhir/*; do curl -X POST -H "Content-Type: application/fhir+json;charset=utf-8" --data @$FILE http://localhost:8098/fhir ; done

and for GCP (only the curl command is different as we need to add the auth token):

for FILE in output/fhir/hospital*.json; do curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/fhir+json;charset=utf-8" --data-binary @$FILE https://healthcare.googleapis.com/v1beta1/projects/google.com:nerdynerd/locations/us-central1/datasets/fhir/fhirStores/test/fhir  ; done

for FILE in output/fhir/practitioner*.json; do curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/fhir+json;charset=utf-8" --data-binary @$FILE https://healthcare.googleapis.com/v1beta1/projects/google.com:nerdynerd/locations/us-central1/datasets/fhir/fhirStores/test/fhir  ; done

for FILE in output/fhir/*; do curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/fhir+json;charset=utf-8" --data @$FILE https://healthcare.googleapis.com/v1beta1/projects/google.com:nerdynerd/locations/us-central1/datasets/fhir/fhirStores/test/fhir  ; done
omarismail94 commented 3 years ago

Update:

Disclaimer: Im still learning the terminology, feel free to correct me or seek clarification if something I typed makes no sense :D

Using the Synthea Model Generator tool, I've created a sample model incorporating some of the Q&As in the table above. A pictorial view of the model is here, and all the JSONs used to create the model are here. Here is some sample data from the model.

There is a limitation of the Synthea Model Generator. It only accepts LOINC codes as keys for Observations, and SNOMED codes for Values. I haven't done the mapping from CIEL to LOINC and SNOMED, nor have I added all the answers yet as I just wanted to see if the model works.

From the Q&A table above, it seems like some questions have nested questions in the answer column. For example: the answers in the question TESTS ORDERED seem like questions where the answer is True/False, e.g. The value for COMPLETE BLOOD COUNT (1019) can be True, HEMOGLOBIN (21) can be False, etc. I have modelled nested questions as MultiObservations, where the answer will be a number in the range of 0-1. Pictorial view can be seen here.

Having nested questions gives us two challenges. First, post-processing will need to be done to round the component.value.quantity.value field to 0 or 1; the range generator outputs a FLOAT between 0 and 1, so we would need to round up/down the values to get 0 or 1. The second challenge is that having nested questions means using more columns from the Observation table. To get the answers to the nested questions, we have to unnest the component and component.code.coding fields, in addition to value.codeableConcept.coding and code.coding. I uploaded the data BigQuery via GCP FHIR Store, and the query to get a summary of the results would look something like in [1]

I guess the question becomes: do we need to have the outermost question. E.g. can we drop TESTS ORDERED from the model and just have each nested question be its own individual question, with Yes/No as the answer?

[1]

SELECT
  coding,
  obs.value.quantity.value AS value,
  valueCoding,
  comp_cod.display,
  IF(comp.value.quantity.unit = "bool", ROUND(comp.value.quantity.value),comp.value.quantity.value ),
  comp.value.quantity.unit,
  subject.patientId,
  effective.dateTime
FROM
  `google.com:nerdynerd.death.Observation` obs
CROSS JOIN
  UNNEST(obs.code.coding) coding
LEFT JOIN UNNEST(component) comp
LEFT JOIN UNNEST(comp.code.coding) comp_cod
LEFT JOIN
  UNNEST(obs.value.codeableConcept.coding) valueCoding
kimaina commented 3 years ago

Thank you @omarismail94 for this incredible work and step by step explanation. The model is really easy to follow and emulates the standard care treatment workflows from enrollment to exit. Looks like you've made so much progress!

{
    "entry": [

        {
            "fullUrl": "/omrs/ws/fhir2/R4/Observation/some-order-uuid",
            "resource": {
                "resourceType": "Observation",
                "id": "some-order-uuid",
                "meta": {
                    "tag": [
                        {
                            "system": "http://terminology.hl7.org/CodeSystem/v3-ObservationValue",
                            "code": "SUBSETTED",
                            "display": "Resource encoded in summary mode"
                        }
                    ]
                },
                "status": "final",
                "code": {
                    "coding": [
                        {
                            "code": "a89c2268-1350-11df-a1f1-0026b9348838",
                            "display": "TESTS ORDERED"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                            "code": "1271",
                            "display": "TESTS ORDERED"
                        },
                        {
                            "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                            "code": "1271",
                            "display": "TESTS ORDERED"
                        }
                    ]
                },
                "subject": {
                    "reference": "Patient/sample-test=patient",
                    "type": "Patient",
                    "display": "TEST TEST"
                },
                "encounter": {
                    "reference": "Encounter/some-enc-id",
                    "type": "Encounter"
                },
                "effectiveDateTime": "2022-01-14T00:00:00+03:00",
                "issued": "2022-01-15T21:16:46.000+03:00",
                "valueCodeableConcept": {
                    "coding": [
                        {
                            "code": "a897e450-1350-11df-a1f1-0026b9348838",
                            "display": "SERUM CREATININE"
                        },
                        {
                            "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                            "code": "790",
                            "display": "SERUM CREATININE"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                            "code": "790",
                            "display": "SERUM CREATININE"
                        }
                    ]
                }
            }
        },
        {
            "fullUrl": "/omrs/ws/fhir2/R4/Observation/some-order-uuid2",
            "resource": {
                "resourceType": "Observation",
                "id": "some-order-uuid2",
                "meta": {
                    "tag": [
                        {
                            "system": "http://terminology.hl7.org/CodeSystem/v3-ObservationValue",
                            "code": "SUBSETTED",
                            "display": "Resource encoded in summary mode"
                        }
                    ]
                },
                "status": "final",
                "code": {
                    "coding": [
                        {
                            "code": "a89c2268-1350-11df-a1f1-0026b9348838",
                            "display": "TESTS ORDERED"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                            "code": "1271",
                            "display": "TESTS ORDERED"
                        },
                        {
                            "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                            "code": "1271",
                            "display": "TESTS ORDERED"
                        }
                    ]
                },
                "subject": {
                    "reference": "Patient/sample-test=patient",
                    "type": "Patient",
                    "display": "TEST TEST"
                },
                "encounter": {
                    "reference": "Encounter/some-enc-id",
                    "type": "Encounter"
                },
                "effectiveDateTime": "2022-01-14T00:00:00+03:00",
                "issued": "2022-01-15T21:16:46.000+03:00",
                "valueCodeableConcept": {
                    "coding": [
                        {
                            "code": "a89a7418-1350-11df-a1f1-0026b9348838",
                            "display": "SERUM POTASSIUM"
                        },
                        {
                            "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                            "code": "1133",
                            "display": "SERUM POTASSIUM"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                            "code": "1133",
                            "display": "SERUM POTASSIUM"
                        }
                    ]
                }
            }
        },

{
            "fullUrl": "/omrs/ws/fhir2/R4/Observation/some-uuid-pulse",
            "resource": {
                "resourceType": "Observation",
                "id": "some-uuid-pulse",
                "meta": {
                    "tag": [
                        {
                            "system": "http://terminology.hl7.org/CodeSystem/v3-ObservationValue",
                            "code": "SUBSETTED",
                            "display": "Resource encoded in summary mode"
                        }
                    ]
                },
                "status": "final",
                "category": [
                    {
                        "coding": [
                            {
                                "system": "http://terminology.hl7.org/CodeSystem/observation-category",
                                "code": "laboratory",
                                "display": "Laboratory"
                            }
                        ]
                    }
                ],
                "code": {
                    "coding": [
                        {
                            "code": "a8a65f12-1350-11df-a1f1-0026b9348838",
                            "display": "PULSE"
                        },
                        {
                            "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                            "code": "5087",
                            "display": "PULSE"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                            "code": "5087",
                            "display": "PULSE"
                        }
                    ]
                },
                "subject": {
                    "reference": "Patient/sample-test=patient",
                    "type": "Patient",
                    "display": "TEST TEST"
                },
                "encounter": {
                    "reference": "Encounter/some-enc-id",
                    "type": "Encounter"
                },
                "effectiveDateTime": "2022-01-14T00:00:00+03:00",
                "issued": "2022-01-15T21:16:46.000+03:00",
                "valueQuantity": {
                    "value": 119,
                    "unit": "rate/min"
                },
                "referenceRange": [
                    {
                        "low": {
                            "value": 0
                        },
                        "high": {
                            "value": 230
                        },
                        "type": {
                            "coding": [
                                {
                                    "system": "http://fhir.openmrs.org/ext/obs/reference-range",
                                    "code": "absolute"
                                }
                            ]
                        }
                    }
                ]
            }
        },

        {
            "fullUrl": "/omrs/ws/fhir2/R4/Observation/some-HPx-pulse",
            "resource": {
                "resourceType": "Observation",
                "id": "some-HPx-pulse",
                "meta": {
                    "tag": [
                        {
                            "system": "http://terminology.hl7.org/CodeSystem/v3-ObservationValue",
                            "code": "SUBSETTED",
                            "display": "Resource encoded in summary mode"
                        }
                    ]
                },
                "status": "final",
                "code": {
                    "coding": [
                        {
                            "code": "7c4a06ce-11eb-496f-b4db-516ab8d78aa8",
                            "display": "HYPERTENSION TREATMENT STARTED"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                            "code": "7332",
                            "display": "HYPERTENSION TREATMENT STARTED"
                        }
                    ]
                },
                "subject": {
                    "reference": "Patient/sample-test=patient",
                    "type": "Patient",
                    "display": "TEST TEST"
                },
                "encounter": {
                    "reference": "Encounter/some-enc-id",
                    "type": "Encounter"
                },
                "effectiveDateTime": "2022-01-14T00:00:00+03:00",
                "issued": "2022-01-15T21:16:46.000+03:00",
                "valueCodeableConcept": {
                    "coding": [
                        {
                            "code": "a8a43750-1350-11df-a1f1-0026b9348838",
                            "display": "LOSARTAN"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                            "code": "2265",
                            "display": "LOSARTAN"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/NLM/sources/RxNORM/",
                            "code": "52175",
                            "display": "LOSARTAN"
                        },
                        {
                            "system": "https://app.openconceptlab.org/#/orgs/IHTSDO/sources/SNOMED-CT/",
                            "code": "96309000",
                            "display": "LOSARTAN"
                        },
                        {
                            "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                            "code": "79074",
                            "display": "LOSARTAN"
                        }
                    ]
                }
            }
        },

    ]
}
kimaina commented 3 years ago

@omarismail94 Instead of

can we drop TESTS ORDERED from the model and just have each nested question be its own individual question, with Yes/No as the answer?

Is it possible to :

have tests ordered in the model and have response as the individual test code (doesn't have to be nested) , like in the shared JSON above

kimaina commented 3 years ago

Lastly

I'm not sure how Synthea works, is it possible to change all multiObs types to normal obs with multiple answer codes / obs with hasMember for example

can this:

"PATIENT REPORTED CURRENT TB TREATMENT (1111)": {
      "type": "MultiObservation",
      "category": "survey",
      "number_of_observations": 0,
      "codes": [
        {
          "system": "LOINC",
          "code": 1111,
          "display": "PATIENT REPORTED CURRENT TB TREATMENT (1111)",
          "value_set": ""
        }
      ],
      "direct_transition": "PATIENT REPORTED CURRENT ANTIRETROVIRAL TREATMENT (2154)",
      "observations": [
        {
          "category": "survey",
          "unit": "bool",
          "codes": [
            {
              "system": "LOINC",
              "code": 5829,
              "display": "PYRAZINAMIDE"
            }
          ],
          "range": {
            "low": 0,
            "high": 1
          }
        },
        {
          "category": "survey",
          "unit": "bool",
          "codes": [
            {
              "system": "LOINC",
              "code": 438,
              "display": "STREPTOMYCIN"
            }
          ],
          "range": {
            "low": 0,
            "high": 1
          }
        },
        {
          "category": "survey",
          "unit": "bool",
          "codes": [
            {
              "system": "LOINC",
              "code": 1131,
              "display": "RIFAMPICIN ISONIAZID PYRAZINAMIDE AND ETHAMBUTOL"
            }
          ],
          "range": {
            "low": 0,
            "high": 1
          }
        }
      ],
      "name": "PATIENT REPORTED CURRENT TB TREATMENT (1111)"
    },

be changed to 


{
    "fullUrl": "/omrss/ws/fhir2/R4/Observation/some-id",
    "resource": {
        "resourceType": "Observation",
        "id": "some-id",
        "meta": {
            "tag": [
                {
                    "system": "http://terminology.hl7.org/CodeSystem/v3-ObservationValue",
                    "code": "SUBSETTED",
                    "display": "Resource encoded in summary mode"
                }
            ]
        },
        "status": "final",
        "code": {
            "coding": [
                {
                    "code": "a899e444-1350-11df-a1f1-0026b9348838",
                    "display": "PATIENT REPORTED CURRENT TUBERCULOSIS TREATMENT"
                },
                {
                    "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                    "code": "1111",
                    "display": "PATIENT REPORTED CURRENT TUBERCULOSIS TREATMENT"
                },
                {
                    "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                    "code": "1111",
                    "display": "PATIENT REPORTED CURRENT TUBERCULOSIS TREATMENT"
                }
            ]
        },
        "subject": {
            "reference": "some-pt-uuid2",
            "type": "Patient",
            "display": "NAME NAME"
        },
        "encounter": {
            "reference": "Encounter/some-id",
            "type": "Encounter"
        },
        "effectiveDateTime": "2025-08-04T12:46:36+03:00",
        "issued": "2025-08-04T12:51:03.000+03:00",
        "valueCodeableConcept": {
            "coding": [
                {
                    "code": "a899f51a-1350-11df-a1f1-0026b9348838",
                    "display": "RIFAMPICIN ISONIAZID PYRAZINAMIDE AND ETHAMBUTOL"
                },
                {
                    "system": "https://openconceptlab.org/orgs/CIEL/sources/CIEL",
                    "code": "1131",
                    "display": "RIFAMPICIN ISONIAZID PYRAZINAMIDE AND ETHAMBUTOL"
                },
                {
                    "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                    "code": "1131",
                    "display": "RIFAMPICIN ISONIAZID PYRAZINAMIDE AND ETHAMBUTOL"
                }
            ]
        }
    }
},

OR


{
    "fullUrl": "/omrss/ws/fhir2/R4/Observation/one",
    "resource": {
        "resourceType": "Observation",
        "id": "csome uidd",
        "meta": {
            "tag": [
                {
                    "system": "http://terminology.hl7.org/CodeSystem/v3-ObservationValue",
                    "code": "SUBSETTED",
                    "display": "Resource encoded in summary mode"
                }
            ]
        },
        "status": "final",
        "code": {
            "coding": [
                {
                    "code": "a8a072c8-1350-11df-a1f1-0026b9348838",
                    "display": "PATIENT REPORTED CURRENT TUBERCULOSIS TREATMENT"
                },
                {
                    "system": "https://app.openconceptlab.org/#/orgs/AMPATH/sources/AMPATH/",
                    "code": "1131",
                    "display": "PATIENT REPORTED CURRENT TUBERCULOSIS TREATMENT"
                }
            ]
        },
        "subject": {
            "reference": "Patient/---------------",
            "type": "Patient",
            "display": "Name 2"
        },
        "encounter": {
            "reference": "Encounter/xxxx",
            "type": "Encounter"
        },
        "effectiveDateTime": "2025-08-02T00:00:00+03:00",
        "issued": "2025-08-12T15:08:22.000+03:00",
        "hasMember": [
            {
                "reference": "Observation/FOR CURRENT TUBERCULOSIS TREATMENT (PYRAZINAMIDE)",
                "type": "Observation"
            },
            {
                "reference": "Observation/FOR CURRENT TUBERCULOSIS TREATMENT (RIFAMPICIN)",
                "type": "Observation"
            },
            {
                "reference": "Observation/FOR CURRENT TUBERCULOSIS TREATMENT (ETHAMBUTOL)",
                "type": "Observation"
            }
        ]
    }
},
ibacher commented 3 years ago

I'm surprised Synthea only supports LOINC for Questions and SNOMED for answers

With Synthea, it's important to remember that it was created out of the need to generate synthetic health-care data to represent patients in the state of Massachusetts (where it was used to test the Massachusetts HIE), so it sometimes makes some very US-centric, HIE-centric assumptions. Using LOINC (for questions) and SNOMED (for answers) is basically the standard practice in the US.

kimaina commented 3 years ago

Hi @omarismail94 please find the attached:

enc_types.csv

bashir2 commented 2 years ago

@omarismail94 can we close this or is there anything left from this issue to be done?

omarismail94 commented 2 years ago

We can close it! Thanks 😃