sunlabuiuc / PyHealth

A Deep Learning Python Toolkit for Healthcare Applications.
https://pyhealth.readthedocs.io
MIT License
956 stars 207 forks source link

HALO Synthetic Data Generation #165

Closed BPDanek closed 1 year ago

BPDanek commented 1 year ago

implement model based on: https://pubmed.ncbi.nlm.nih.gov/36945542/

BPDanek commented 1 year ago

output statistics:

(pyhealth) [bdanek2@sunlab-work-02 PyHealth]$ /home/bdanek2/miniconda3/envs/pyhealth/bin/python /home/bdanek2/PyHealth/pyhealth/models/halo.py
training time: 4.7031790018081665e-07 7.838631669680277e-09 1.3064386116133796e-10
Validation Loss:0.0241066
Generating samples for labels: 100%|███████████████████████████████████████████████████████████████████████████████████| 2/2 [00:17<00:00,  8.88s/it]
Saving synthetic ehr dataset at: /home/bdanek2/PyHealth/synthetically_generated_data.pkl                                                             
source (train)
{
        Record Length Mean: 1.3559782608695652,
        Record Length Standard Deviation: 0.6349573073334189,
        Visit Length Mean: 29.312625250501004,
        Visit Length Standard Deviation: 19.350745816354003,
        Inter-visit time Mean: 29.312625250501004,
        Inter-visit time Standard Deviation: 19.350745816354003,
}
{
        (0.0,): 1.0,
        (1.0,): 1.0,
        (1.0, 1.0): 1.0,
}
synthetic
{
        Record Length Mean: 1.2519,
        Record Length Standard Deviation: 0.4861546976014939,
        Visit Length Mean: 24.54501158239476,
        Visit Length Standard Deviation: 14.314390667157445,
        Inter-visit time Mean: 24.54501158239476,
        Inter-visit time Standard Deviation: 14.314390667157445,
}
{
        (0.0,): 1.0,
        (1.0,): 1.0,
        (1.0, 1.0): 1.0,
}
Evalutor: generating label plots:   0%|                                                                                        | 0/2 [00:00<?, ?it/s]
Label stats ./halo_eval_plots_Per Record Code Probabilities_male:
Per Record Code Probabilities r-squared = 0.9540811188059296

Label stats ./halo_eval_plots_Per Visit Code Probabilities_male:
Per Visit Code Probabilities r-squared = 0.9641675839182102

Label stats ./halo_eval_plots_Per Record Bigram Probabilities_male:
Per Record Bigram Probabilities r-squared = 0.8302730946294461

Label stats ./halo_eval_plots_Per Visit Bigram Probabilities_male:
Per Visit Bigram Probabilities r-squared = 0.8527011441652503

Label stats ./halo_eval_plots_Per Record Sequential Visit Bigram Probabilities_male:
Per Record Sequential Visit Bigram Probabilities r-squared = 0.3990111980028218

Label stats ./halo_eval_plots_Per Visit Sequential Visit Bigram Probabilities_male:
Per Visit Sequential Visit Bigram Probabilities r-squared = 0.4897344982576297
Evalutor: generating label plots:  50%|████████████████████████████████████████                                        | 1/2 [00:02<00:02,  2.53s/it]
Label stats ./halo_eval_plots_Per Record Code Probabilities_female:
Per Record Code Probabilities r-squared = 0.9540811188059296

Label stats ./halo_eval_plots_Per Visit Code Probabilities_female:
Per Visit Code Probabilities r-squared = 0.9641675839182102

Label stats ./halo_eval_plots_Per Record Bigram Probabilities_female:
Per Record Bigram Probabilities r-squared = 0.8302730946294461

Label stats ./halo_eval_plots_Per Visit Bigram Probabilities_female:
Per Visit Bigram Probabilities r-squared = 0.8527011441652503

Label stats ./halo_eval_plots_Per Record Sequential Visit Bigram Probabilities_female:
Per Record Sequential Visit Bigram Probabilities r-squared = 0.3990111980028218

Label stats ./halo_eval_plots_Per Visit Sequential Visit Bigram Probabilities_female:
Per Visit Sequential Visit Bigram Probabilities r-squared = 0.4897344982576297
Evalutor: generating label plots: 100%|████████████████████████████████████████████████████████████████████████████████| 2/2 [00:05<00:00,  2.55s/it]
plots at: ./halo_eval_plots_Per Record Code Probabilities_male
./halo_eval_plots_Per Visit Code Probabilities_male
./halo_eval_plots_Per Record Bigram Probabilities_male
./halo_eval_plots_Per Visit Bigram Probabilities_male
./halo_eval_plots_Per Record Sequential Visit Bigram Probabilities_male
./halo_eval_plots_Per Visit Sequential Visit Bigram Probabilities_male
./halo_eval_plots_Per Record Code Probabilities_female
./halo_eval_plots_Per Visit Code Probabilities_female
./halo_eval_plots_Per Record Bigram Probabilities_female
./halo_eval_plots_Per Visit Bigram Probabilities_female
./halo_eval_plots_Per Record Sequential Visit Bigram Probabilities_female
./halo_eval_plots_Per Visit Sequential Visit Bigram Probabilities_female
done