Open hitvoice opened 4 years ago
We basically use the following setting: lr=2e-5 batch_size=32, max_epochs=3 To reproduce the results on CLUE, we also need some other techniques which has been listed on CLUE leaderboard. We will open-source related code in the near future.
Got it. Thanks for answering!
Recently you updated "BERT pretrained on mixed large Chinese corpus (bert-large 24-layers) " on ReadMe. What hyperparameters (lr, batch size, max epochs) did you use when fine-tuning on CLUE?