search

deeppavlov / deeppavlov-gsoc-ideas

4 stars 7 forks source link

refactor Multitask BERT #5

Closed oserikov closed 2 years ago

oserikov commented 3 years ago

difficulty: medium mentor: @dimakarp1996
requirements: python, ML useful links: Multitask BERT

Idea description

Implement at least one of the following tasks: PAL-BERT, CA-MTL, MT-DNN(more is better) within the DeepPavlov Library

Coding Challenge

Reproduce results on the GLUE benchmark, which are described in the articles https://arxiv.org/pdf/2009.09139 https://arxiv.org/abs/1901.11504 https://arxiv.org/abs/1902.02671 at least for 1 paper (more is better) using Pytorch framework

rimijoker commented 3 years ago

@dimakarp1996 @oserikov Is this project still being considered for GSoC 2021? I was going through the CA-MTL paper in detail yesterday and found it really fascinating, It would be an exciting opportunity to implement this within the DeepPavlov Library.

dimakarp1996 commented 3 years ago

@rimijoker yes, it is, keep calm