Training Parameters for BERT classification on MSMarco

koursaros-ai / nboost

NBoost is a scalable, search-api-boosting platform for deploying transformer models to improve the relevance of search results on different platforms (i.e. Elasticsearch)

Apache License 2.0

675 stars 69 forks source link

Training Parameters for BERT classification on MSMarco #99

Open gsgoncalves opened 3 years ago

gsgoncalves commented 3 years ago

Hi,

Congrats on your project!

Could you please share the training details that lead to the BERT models finetuned on MSMarco. I am especially interested in the learning rate, batch size, and the number of steps.

Did you follow all the specifications given by Nogueira and Cho? I could see that this is reference number 2 in your benchmarks section. However, the text that refers to this reference seems to be missing.

Thank you

davidmrau commented 2 years ago

I'd be also interested in that!