请问您的ptuning是V1吗，现在支持ptuning V2吗，不知道哪个在生成上好一点。您的几个微调方法lora，adalora，ptuning哪个效果好啊，full-finetuning一把学习率是多少啊～

shibing624 / textgen

TextGen: Implementation of Text Generation models, include LLaMA, BLOOM, GPT2, BART, T5, SongNet and so on. 文本生成模型，实现了包括LLaMA，ChatGLM，BLOOM，GPT2，Seq2Seq，BART，T5，UDA等模型的训练和预测，开箱即用。

Apache License 2.0

935 stars 109 forks source link

请问您的ptuning是V1吗，现在支持ptuning V2吗，不知道哪个在生成上好一点。您的几个微调方法lora，adalora，ptuning哪个效果好啊，full-finetuning一把学习率是多少啊～ #30

Open hangzeli08 opened 1 year ago

hangzeli08 commented 1 year ago

Describe the Question

Please provide a clear and concise description of what the question is.

Describe your attempts

[ ] I walked through the tutorials
[ ] I checked the documentation
[ ] I checked to make sure that this is not a duplicate question

shibing624 commented 1 year ago

推荐用lora，最新方法是adalora，对低资源的方法更友好，且保存的文件大小更小，10MB以内。

shibing624 commented 1 year ago

学习率lr =2e-5

stale[bot] commented 10 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.(由于长期不活动，机器人自动关闭此问题，如果需要欢迎提问)