Pneumonia Risk predictiON neTwOrk (PRONTO)

A TensorFlow model for predicting imminent pneumonia risk with a given time window based on retrospective analysis of longitudinal clinical notes. Note: a more general, extended version of this project is available here.

Dependencies

• Python >= 3.6
• TensorFlow >= 1.9
• numpy >= 1.13.3
• scikit-learn >= 0.19.0
• scipy >= 0.13.3

Installation

First, install the required dependencies:

$ pip -r requirements.txt

Then, install TensorFlow with or without gpu support:

Optionally install any of the below optional dependencies:

Training and Evaluating PRONTO

To train and evaluate PRONTO, you need to pass pronto a path to a chronology file and a vocabulary file.

export CHRONOLOGY_FILE=/path/to/chronology.tsv
export VOCAB_FILE=/path/to/vocabulary.tsv
export OUTPUT_DIR=/path/to/output/directory

python run_pronto.py \                                                                                                                   
      --chrono_file=$CHRONOLOGY_FILE  \
      --vocab_file=$VOCAB_FILE \
      --output_dir=$OUTPUT_DIR \
      --max_snapshot_size 256 \
      --batch_size=32 \
      --dropout=0.00 \
      --vocab_dropout=0.50 \
      --num_epochs 20 \
      --print_performance

Required command-line arguments:

Additional command-line arguments:

Convenience options:

Super secret options:

Training and Evaluating Support Vector Machines

Training and evaluating SVMs can be accomplished by:

$ python -m run_svm.py [-h] \
    --chronology-path $CHRONOLOGY_FILE \
    --vocabulary-path $VOCAB_FILE \
    --final-only 

Required command-line arguments:

Additional command-line arguments:

Training and Evaluating Baselines

Training and evaluating baselines can be accomplished by:

$ python -m run_baselines.py [-h] \
    --chronology-path $CHRONOLOGY_FILE \
    --vocabulary-path $VOCAB_FILE \
    --final-only 

Required command-line arguments:

Additional command-line arguments:

Data format

The pronto script load chronology and vocabulary files. Chronology and vocabulary files are assumed to follow specific formats.

Chronology format

The format of this chronology file is assumed to be as follows:

[external_patient_id]:[external_visit_id]\t[chronology]

where each [chronology] is encoded as as sequence of snapshots, separated by tabs:

[[snapshot]\t...]

and each [snapshot] is encoded as:

[delta] [label] [observation IDs..]

Note that snapshots are delimited by spaces, label must be 'true' or 'false', delta is represented in seconds since previous chronology, and observation IDs should be the IDs associated with the observation in the given vocabulary file.

For example, the line:

11100004:1108a   0 false 1104 1105 2300 25001    86400 false 1104 2300   172800 true 1104 2300 3500

would indicate that patient with external ID '11100004' and visit with external ID '1108a' had a chronology including 3 snapshots where:

• the first snapshot was negative for pneumonia, had a delta of 0, and contained only three clinical observations: those associated with vocabulary terms 1104, 1105, 2300, and 25001;
• the second snapshot was negative for pneumonia, had a delta of 86400 seconds (1 day), and included only two clinical observations: 1104 and 2300
• the third snapshot was positive for pneumonia, had a delta of 172800 seconds (2 days), and included only three clinical observations: 1104, 2300, and 3500

Vocabulary format

The vocabulary file is assumed to be formatted as follows:

[observation]\t[frequency]

where the line number indicates the ID of the observation in the chronology (e.g., 1104), [observation] is a human-readable string describing the observation, and [frequency] is the frequency of that observation in the dataset (this value is only important if specifying a max_vocabulary_size as terms will be sorted in descending frequency before the cut-off is made)

Note: as described in the AMIA paper, chronologies are truncated to terminate at the first positive label. Chronologies in which the first snapshot is positive or in which no snapshot is positive are discarded. The PRONTO scripts will automatically perform the truncation for you.