Semi-Supervised Learning of the Electronic Health Record for Phenotype Stratification

Overview:

Used electronic health record (EHR) data from the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) to predict ALS patient survival time. Leveraged denoising autoencoders (DA) to enhance the prediction performance of a random forest (RF) model.

Biology & Background:

Discusses the importance of disease phenotyping.
- Provides historical basis for this claim.
- Subtyping "may contribute to additional physiological understanding".
Analysis of EHRs may improve traditional studies of disease.
Though vast, EHRs are a largely untapped resource.
- EHRs were designed for billing, not research.
- EHRs are often noisy, sparse or incomplete and "high quality phenotyping is costly and requires physician review".
- Supervised models do not fully consider the available unlabeled data, and it is not cost effective to label the data.
- "Previous work in semi-supervised learning of the EHR relies on closed source commercial software [19], and natural language processing of free text fields to match clinical diagnosis [20, 21]."
The PRO-ACT data set contains EHR data for 23 clinical trials with a total of 10723 ALS patients. However, death information is only available for 3398 of the patients.
- This data set is "relatively homogenous" as it is from clinical trials. The authors recognize that additional research may be needed to adapt their methods to "diverse multi-disease real world clinical data".
- This data set does not include free text and image data.

Computational aspects:

DA+RF performance was evaluated on both real (PRO-ACT) and simulated data.

Simulation:

For each simulation, a simulation-specific subset (1, 2, 3, 8, or 16) of features was chosen to be altered via "hidden input effects".
- Note that, though differing with each simulation, this subset is consistent across all patients.
- For brevity's sake, I have not described how this data is generated. While significant to the paper, it may be less relevant to our focus on the applications of deep learning. However, note that they consider the cases of missing data and non-binary labels.
The DA + RF was compared against traditional classifiers + mean imputation.
- Traditional classifiers: nearest neighbors (with k of 5) (NN), RF (with 10 estimators), support vector machine (with radial basis function kernel) (SVM).
- "10-fold cross validation across 10 independent replicates for each set of parameters".
- At present, model performance (across all simulations) with DA preprocessing is only available for the RF.
Evaluated DA performance with varying number of hidden nodes (1, 2, 4, and 8).
Supervised classification comparison:
- Simulate real noise by adding a systematic bias of 0.1 applied to 33% of patients.
- Used (random and normally distributed) features vectors of various sizes (50, 100, 200, and 400).
- Considered data sets of varying sizes (100, 200, 500, 1000, 2000)`.
- Even split of case`/control patients.
- labels were stripped from data during unsupervised pre-training.
- DA+RF with 2 nodes performed best, but still not much better than RF alone.
- Performance data not yet available for DA+RF with more than 2 nodes.
Semi-supervised classification comparison:
- Used an additional 10000 simulated patients (with 1:1 case-control ratio) during unsupervised pre-training.
- No systematic bias, but otherwise the same as the supervised classification comparison.
Missing data comparison:
- Repeated the semi-supervised classification comparison with 0%, 10%, 20%, 30%, and 40% of data missing.
- Modified DA's cost function of to exclude missing data (so as to avoid imputation of values).
- DA+RF appears to do best, though all models do poorly with a large portion of missing data (i.e. 40% or more) (especially with the non-binary labels).
Clustering and visualization:
- Compared clustering (with PCA + t-SNE) of the DA's hidden nodes with that of the raw data.
- Varied number of hidden nodes (2, 3, and 4). More nodes appeared to result in cleaner clustering.

PRO-ACT data:

Performed unsupervised training of the DA with the full data set (10723 patients) before performing supervised training (on labeled 3398 patients) with the random forest.
- Input vector of 6812 features.
- 10-fold cross validation.
- Standardized scales and normalized features to fall between 0 and 1
Compared DA+RF against NN+RF
- DA had 250 hidden nodes, and trained for 1000 epochs.
- NN has k of 15.
- DA+RF has lower mean absolute error (appears to be 15 to 20 days lower).
Evaluated clustering (with PCA + t-SNE) of raw inputs against that of hidden nodes.
- Hidden nodes appear to cluster better by survival time.

Takeaway:

Performance of the DA+RF is compared against multiple traditional models.
The paper carefully explores a range of model hyperparameters in a controlled manner. This gives a clear and logical basis for the model used to classify the clinical data.
Aside from the unsupervised pre-training, the DA required no additional preprocessing. However, the DA still improved the performance of RFs (and clustering). This strategy, though simple, could also improve prediction performance for other problems (especially those for which most data is unlabeled).

greenelab / deep-review