agitter
commented
8 years ago Biology
- Tox21 Data Challenge: 12000 chemicals screened with 12 assays
- Assays are for toxicity, which is related to other virtual screening tasks but different from predicting the effects for a single protein target (#56)
- Challenge is to predict effects of 647 test compounds; challenge organizers split compounds into training, leaderboard, and test sets
- Good overview of similarity-based and feature-based algorithms for predicting chemical effects
- Compound can be active, inactive, or inconclusive/untested for each assay; many more inactives than actives
- Input: chemical compound in SDF format (graph of atoms and their bonds), transforms these into chemical descriptors (as in #57) and molecular fingerprints (as in #55); ultimately using more features than related work
Computational aspects
- Cites graph-based kernels for molecular graphs dating back to 2003 (relevant to #52 and #53)
- Acknowledge some of the related deep learning methods but do not discuss them or compare to them
- Won the grand challenge and several sub-challenges, but not the top performer for every individual assay
- Score with AUC despite class imbalance
- Use multi-task learning, number of tasks is comparable to (#57) but less than (#55)
- Claim deep learning in general performs well with a large dataset, related input features, and multi-task setting. Tox21 satisfies these conditions.
- Indicator variable in objective function to ignore untested compounds
- ReLU hidden units with multi-task sigmoid output layer
- 1024, 2048, 4096, 8192, or 16384 hidden units per layer and up to 4 hidden layers
- Optimize hyperparameters separately for each task even though they have a multi-task network
- Supplement the challenge training data with similar compounds and assays from PubChem, ChEMBL, etc.
- Also use SVM, random forest, and elastic nets both as competing methods and in an ensemble with the neural network
- Cluster-based cross validation to help prevent indirect test fold leakage when compounds are very similar
- Multi-task network outperforms single-task for 10 of 12 assays, but #55 has a better assessment of the impact of the number of tasks in this domain
- Adding chemical descriptor inputs doesn't substantially boost performance relative to fingerprint features alone
- For interpretation, associate hidden units with toxicophores using U-test and correlation
- They claim that the neural network is better than competing methods in most cases, but either SV or random forest is better than or at least competitive with the neural network in terms of AUC for many of the assays
- Provide code and a cleaned dataset
Why include it in the review
- Winning a challenge can help bolster claims that this is the state of the art in the domain
- This is not the authors' interpretation, but in my opinion the AUCs may suggest that the neural network was more of an incremental than a 'transformative' improvement for many assays
- Includes some discussion of interpreting hidden units in the virtual screening domain
- The study is generally well-executed, even if many of the computational ideas had already appeared in other virtual screening papers. This work could be mentioned but may not be a major focal point.
Other notes
- They cite http://arxiv.org/abs/1503.01445 which has a slightly different title and author order but appears to be the preprint version of this paper
hmf0103
vinay-hebb
http://doi.org/10.3389/fenvs.2015.00080
Related to virtual screening #45.