Multitask and few-shot learning

[jaclyn-taroni]

In the interest of getting #63 in and discussion for this afternoon's (my time anyway 😄 ) meeting, I'm filing an issue to specifically track the addition of multitask and few-shot learning to the prior knowledge section of the manuscript.

Here's the stub that was added in https://github.com/jaybee84/ml-in-rd/pull/63/commits/89a83b95bf5bad9438cbb0c63ec14dac1c8e8dc1:

Multitask and few-shot learning

Where transfer learning can be supervised or unsupervised, the related approaches multitask and few-shot learning are forms of supervised learning. Multitask learning is an approach where classifiers are learned for related tasks at the same time using a shared representation [@doi:10.1023/A:1007379606734]. Few-shot learning is the generalization of a model trained on related data and tasks to a new task with limited labeled data (e.g., the detection of a motorcycle from a low number of examples of a motorcycle).

References I expect to cite/discuss

• Liu et al. Multi-task Learning via Adaptation to Similar Tasks for Mortality Prediction of Diverse Rare Diseases arXiv:2004.05318.
• Kearnes, Goldman, and Pande. Modeling Industrial ADMET Data with Multitask Networks arXiv:1606.08793. - I think this is interesting because it explores when multitask learning is beneficial in the context of drug screening and I am keen on the idea of emphasizing that just because an approach seems like a good idea, that's insufficient for it to be a good idea (hopefully folks get that from #63!)
• Caruana. Multitask Learning. doi:10.1023/A:1007379606734.
• Zhang et al. Deep Model Based Transfer and Multi-Task Learning for Biological Image Analysis. doi: 10.1109/TBDATA.2016.2573280 - could be a nice tie-in to the transfer learning section that I expect to precede this section.
• Quellec et al. Automatic detection of rare pathologies in fundus photographs using few-shot learning. doi:10.1016/j.media.2020.101660 - this is the ref mentioned in #45.
• And to include another drug screening type of example: Altae-Tran et al. Low Data Drug Discovery with One-Shot Learning. doi:10.1021/acscentsci.6b00367

[jaclyn-taroni]

I will plan on keeping this brief such that we won't need the 20% reduction pass.