search

nyu-mll / jiant-v1-legacy

The jiant toolkit for general-purpose text understanding models
MIT License
21 stars 9 forks source link

[CLOSED] Reproducing GLUE paper single task results #1110

Closed jeswan closed 4 years ago

jeswan commented 4 years ago

Issue by hsajjad Thursday Aug 20, 2020 at 09:04 GMT Originally opened as https://github.com/nyu-mll/jiant/issues/1110

I am trying to reproduce the results presented in table 6 of GLUE paper. I am mainly interested in single-task bilstm training from scratch which is the first line in the table. I have tried various settings with embeddings trained from scratch or using Glove 840B embedding with single and multi-task but I am unable to produce or even come close to the results presented in table 6.

Do you have a configuration file that can reproduce these results?

jeswan commented 4 years ago

Comment by sleepinyourhat Thursday Aug 20, 2020 at 13:49 GMT

Hi! If you want the exact code from the leaderboard baselines, look here: https://github.com/nyu-mll/GLUE-baselines