Massively Multitask Networks for Drug Discovery

[agitter]

https://arxiv.org/abs/1502.02072

At a glance: Related to virtual screening #45. A supervised learning approach to drug discovery where molecular fingerprints are the input to a multitask classifier. Each classification task is a screening assay. Shows the benefits of multitask learning.

[agitter]

Biology

• Virtual screening domain
• Predict whether a drug-like chemical compound is effective against a particular protein target or in some other type of high-throughput screen
• Inspired by Merck Kaggle competition (cite some criticism of the competition from the community), related to #57 and #75.
• Input: molecular fingerprint of each compound, which is a bit vector
• Output: activity prediction for each compound and each protein target or screening assay
• Does not use protein features (e.g., structure)
• Trained on four datasets: PCBA, MUV, DUD-E, Tox21 (see #72)

Computational aspects

• Uses Keras DistBelief (edited, see below)
• Multitask classification problem where each protein target or screening assay is a task (supervised learning)
• Pyramidal network structure where first hidden layer has 1000, 2000, or 3000 units and second layer has 50, 100, or 150 units
• Multitask network beats logistic regression, random forest, and single task neural networks per AUC and enrichment scores
• Explicitly assess the impact of the number of tasks in the multitask classification problem, e.g., showing how performance improves as more tasks are added and does not plateau even after adding all tasks
• Code not linked in the paper but is available through their deepchem package

Why include it in the review

• In the long term, improving the early screening stages in a drug discovery pipeline could potentially transform how we treat individuals
• Computationally, the thorough analysis of the impact of multitask learning is excellent and something that I wish was discussed in the deep learning regulatory genomics papers; we could suggest this as a best practice for the regulatory genomics community
• There are already reviews of deep learning in bioinformatics and deep learning in virtual screening, but have stayed within their respective disciplines; discussing both could add novelty to the review

[agitter]

@cgreene after you look at this, I'd be interested in discussing whether this type of problem should be in the scope of the review.

[cgreene]

@agitter this seems to be within the scope of the review. Better drugs or a more efficient means of getting to them would seem to play a role in our guiding question.

[agitter]

More Computational aspects

• This is a multilayer perceptron, the input features are unordered
• The output layer has one softmax node per task (protein target or assay)

[agitter]

There is recent related work from some of these authors: http://arxiv.org/abs/1606.08793

I'll have to read it to see if it needs its own issue. The application is ADMET (absorption, distribution, metabolism, excretion, and/or toxicity) assays in an industrial setting.

[rbharath]

@agitter @cgreene Quick note: This work used DistBelief (Google's pre-Tensorflow system) not Keras. Glad to provide any other clarifications on paper you folks need.

[agitter]

Thanks @rbharath, it's great to get direct input from papers' authors. I edited my notes above. You have a lot of expertise in this domain so let me know if you would be interested in seeing a draft of the drug discovery section or helping write some of the review.

[rbharath]

@agitter Sure, I'd be glad to help write part of the review :-). Let me know where I can help out.

[agitter]

@rbharath #188 gives a fairly recent status report about which sections need the most attention. I'd be very interested in your perspective on #174, and we can take our discussion of drug discovery there.

The contributing page and draft intro could also be good overviews before diving in.