search

tloen / alpaca-lora

Instruct-tune LLaMA on consumer hardware
Apache License 2.0
18.42k stars 2.2k forks source link

Do you have any ideas about how to behave after expanding the Chinese vocabulary？ #252

Open 27182812 opened 1 year ago

27182812 commented 1 year ago

I made fine-tuning on belle based on the llama with an expanded Chinese vocabulary, and the effect is polarized. Do you have any good suggestions? My repo: https://github.com/27182812/ChatGLM-LLaMA-chinese-insturct

AngainorDev commented 1 year ago

No idea about chinese myself, but maybe you could check and give guanaco a try

https://huggingface.co/KBlueLeaf/guanaco-7B-leh https://github.com/KohakuBlueleaf/guanaco-lora

This is a fork of this repo addressing chinese and more.

27182812 commented 1 year ago

Wow, thank you!