The Natural Language Decathlon: Multitask Learning as Question Answering

[howardyclo]

Metadata

• Authors: Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, Richard Socher
• Organization: Salesforce Research
• Publish Date: 2018.06
• Paper: https://arxiv.org/pdf/1806.08730.pdf
• Code: https://github.com/salesforce/decaNLP
• Blog: https://einstein.ai/research/blog/the-natural-language-decathlon
• Video: https://www.youtube.com/watch?v=MENYCdm1eis
• Website: http://decanlp.com/

[howardyclo]

Summary

• This paper present a new multitask question answering network (MQAN) that jointly learns all tasks in ten different NLP tasks. (Cast 10 tasks to question answering)
• The model uses dual coattention, multi-head self attention for encoding, and based on pointer-generator network for copying the words from context or question, or generating the words from external vocabulary. No explicit supervision is needed.
• Anti-curriculum learning (learn hard tasks first) >>> curriculum learning (hurt performance)
• The model can perform zero-shot classification tasks due to the unseen new task is represented as questions, and the unseen classes can be copied from the question. (Meta-learning).
• Model and training details are reported.