Request for Hyperparameters to Reproduce Full Fine-Tuning Results in GLUE

QingruZhang / AdaLoRA

AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (ICLR 2023).

MIT License

231 stars 23 forks source link

Request for Hyperparameters to Reproduce Full Fine-Tuning Results in GLUE #1

Closed HarryWu99 closed 1 year ago

HarryWu99 commented 1 year ago

Thanks for your fantastic work!

Now I want to reproduce full Fine-tuning results in Glue using DeBERTaV3-base, but it seems that no specific hyperparameters are given in the paper or in this repo. Specifically, I would like to know the following hyperparameters:

learning rate
weight_decay
random seed

Thanks for your help!

QingruZhang commented 1 year ago

Hello Guanyu, Thanks for your interests in our paper. You can find the specific runing commands to reproduce the NLU results in this folder. Our paper has listed the hyperpapermeters in Appendix C. Please let me know if you get any questions when running these commands or if you are reproducing the results of different target rank. I will post you the specific configs. Thanks, Qingru