GEM-benchmark / GEM-metrics

Automatic metrics for GEM tasks
https://gem-benchmark.com
MIT License
60 stars 20 forks source link

Key hyperparameters to reproduce results? #96

Open VickiCui opened 2 years ago

VickiCui commented 2 years ago

Thanks for your great work! I'm having some trouble when reproducing the results. Although trained as in the tutorial, there is still a big gap between the model I trained and the results given in the paper. Can you provide the necessary hyperparameters in order to reproduce the results? Such as batch size, epoch, learning rate, beam size, and so on. Thx!!!

tuetschek commented 2 years ago

@VickiCui I don't think this is the correct place – the repo here is just for the metrics scripts, it doesn't include the baseline submissions. Perhaps @sebastianGehrmann or @yjernite would know more on the baselines?

VickiCui commented 2 years ago

您好,您的邮件我已经收到,我会第一时间处理。

sebastianGehrmann commented 2 years ago

Which task are we talking about? Most of the models were trained for https://openreview.net/forum?id=CSi1eu_2q96 and the hyperparameters from that paper should replicate results.