uf-hobi-informatics-lab / seqEHR

develop deep learning-based models for learning structured EHR as a sequence
MIT License
5 stars 0 forks source link

seqEHR

develop deep learning-based models for learning structured EHR as sequence

specific model desc

TLSTM

TLSTM first published in Patient Subtyping via Time-Aware LSTM Networks", KDD, 2017

  • We re-implement the TLSTM using PyTorch in TLSTM
  • the implementation was validated using the sync data released in the original TLSTM repo.
  • The results we got on train is AUC of 0.958 and test is AUC of 0.916
  • input data format
    
    # We use a slightly different input format compared to the original implementation.
    # input - three conpoenents: 1. features (OHE or embeddings); 2. time intervals; 3. labels
    # feature - 3D Tensor - [Batch, Sequence, Data]
    # Time - 3D Tensor - [Batch, Sequence, time_diff]
    # label - 2D numpy array - [Batch, label]

Note: in original data released, the time diff data shape is [Batch, time_diff, Sequence] but we reshape this for dimension consistency

see ./TLSTM/test_tlstm.py for more usage details


### LSTM
- This is baseline for comparison purpose
- input data shape

feature - 3D Tensor - [Batch, Sequence, Data]

label - 2D numpy array - [Batch, label]


### Embedding
- we only support pre-trained embeddings
- we do not support in-situ embedding random initialization
- if you want to random initialize embeddings, you have to create a random initialized embeddings yourself
- embedding input shape: ```[Batch, Squence, Features, Embeddings]```. Note that we have 4 dimensions instead of 3 because the each features has been converted to a embedding which is an dense vector whereas the OHE directy using 1-0 to represent features.

### self-attention (TODO)
- we implement a self-attention architecture to replace LSTM and TLSTM
- self-attention proved to be perform better in many NLP tasks over LSTM (seq2seq translation)
- position enmbedding can be used for encode time variance which is suitable for replace TLSTM

## MixStaticSeq model
- we develope a mix model which can handle both static features (e.g., demographics) and time-series features (e.g., diagnoses, medication, procedure, labs)
- we use MLP + TLSTM (or LSTM) as model architecture
- we have a specific data input format doc at: 
- The current evaluation will be measured as ROC-AUC, sensitivity, specificity, and accuracy
- input data format

we expect the input data for MixStaticSeq model contains three parts:

1. static feature

2. sequence features - see TLSTM and LSTM for details

3. labels

static features will be 2-d tensor as [Batch, features]

2 and 3 can be refered to TLSTM or LSTM based on your choice