Multi-Task Deep Neural Networks for Natural Language Understanding

[jinmang2]

집현전 중급반 스터디

• 2022년 7월 3일 일요일 9시
• 이경로님 임승원님 박범수님
• 논문 링크: https://aclanthology.org/P19-1441.pdf

Abstract

In this paper, we present a Multi-Task Deep Neural Network (MT-DNN) for learning representations across multiple natural language understanding (NLU) tasks. MT-DNN not only leverages large amounts of cross-task data, but also benefits from a regularization effect that leads to more general representations in order to adapt to new tasks and domains. MT-DNN extends the model proposed in Liu et al. (2015) by incorporating a pre-trained bidirectional transformer language model, known as BERT (Devlin et al., 2018). MT-DNN obtains new state-of-the-art results on ten NLU tasks, including SNLI, SciTail, and eight out of nine GLUE tasks, pushing the GLUE benchmark to 82.7% (2.2% absolute improvement). We also demonstrate using the SNLI and SciTail datasets that the representations learned by MT-DNN allow domain adaptation with substantially fewer in-domain labels than the pre-trained BERT representations. The code and pre-trained models are publicly available at this https URL.

[Cloud9Bumsu]

https://docs.google.com/presentation/d/1zdwQqVTqoLAfuyAJ2W6uVbiSuehFr3i4/edit?usp=sharing&ouid=106291530285244306793&rtpof=true&sd=true

[jinmang2]

• 발표링크: https://www.youtube.com/watch?v=Oy3Jv9P7i5w&list=PL2tRglRS_GqjCcBVM26gYNdoOnX12lLIh&index=13