Reproduce reported LORA16bit result on GSM8K

yxli2123 / LoftQ

MIT License

180 stars 15 forks source link

Reproduce reported LORA16bit result on GSM8K #9

Closed callanwu closed 6 months ago

callanwu commented 7 months ago

Hello, Great work! I want to reproduce the reported LoRA(16bit) result(36.9) on GSM8K dataset in your paper. Could you provide the correct script or more detailed hyper-parameters? Thx a lot!

yxli2123 commented 6 months ago

Hi @callanwu, please try https://github.com/yxli2123/LoftQ/blob/main/scripts/train_gsm8k.sh#L62

callanwu commented 6 months ago

Thx a lot～ I will try it！