Open ewayuan opened 2 years ago
If you use MultipleNegativesRankingLoss, the batch size is important: the larger, the better.
All other hyper parameters are rather unimportant
If you use MultipleNegativesRankingLoss, the batch size is important: the larger, the better.
All other hyper parameters are rather unimportant
How about the CosineSimilarityLoss? Is the number of epochs important? For some data, 1 epoch is better than 10 epochs, but for another dataset, 10 epochs is better than 1 epoch. How could I find the proper number of epochs?
There a batch size of 32 is sufficient. Check the scores on the dev set to see how long you have to train
After I changed CosineSimilarityLoss to MultipleNegativesRankingLoss, with noDuplicateDataLoader, the performance dropped siginificantly
Hi, I'm planning to continue training bi-encoder based on the all-mpnet-base-v2 model, my own model on my own training dataset, which contains over one million training examples.
Would you give me any suggestion on the number of epochs (Is that higher is better? how many epochs is good for one million training examples?), learning rate and batch size? Maybe also which loss function is better?