Hyperparameters to train stsb-mpnet-base-v2

[XuandongZhao]

I am wondering how can I reproduce the pre-trained models. For example, I think "stsb-mpnet-base-v2" is trained on "ALLNLI" and "STSb" dataset with "MultipleNegativesRankingLoss". Is it right?

Also could you please provide some hyperparameters or scripts you use to train the "stsb-mpnet-base-v2" models, such as "epochs", "learning rate", "batch_size"...

Thank you very much for your help!

[nreimers]

Here the two scripts https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/nli/training_nli_v2.py https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/sts/training_stsbenchmark_continue_training.py

[XuandongZhao]

Here the two scripts https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/nli/training_nli_v2.py https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/sts/training_stsbenchmark_continue_training.py

Thanks, I already read these two scripts. In these two examples, we can see parameters for "distilroberta". Are the hyperparameters all the same for different models?

[nreimers]

Yes

[XuandongZhao]

Yes

Thanks a lot! BTW, could I ask what type of GPU do you use? Is it tesla V100?

[nreimers]

same for me