Facico / Chinese-Vicuna

Chinese-Vicuna: A Chinese Instruction-following LLaMA-based Model —— 一个中文低资源的llama+lora方案,结构参考alpaca
https://github.com/Facico/Chinese-Vicuna
Apache License 2.0
4.14k stars 421 forks source link

if it is possible to fine tune your model #128

Closed TY-hue123 closed 1 year ago

TY-hue123 commented 1 year ago

i tried to change the model's path as MODEL_PATH="Facico/Chinese-Vicuna-lora-7b-3epoch-belle-and-guanaco", but it can't find the model:

https://github.com/Facico/Chinese-Vicuna/blob/master/finetune.sh#L8

image
SHITIANYU-hue commented 1 year ago

I tried to use the pre-trained model (https://huggingface.co/Chinese-Vicuna/Chinese-Vicuna-lora-7b-belle-and-guanaco) to fine-tune on my own dataset but it cannot learn a good talking behavior:

image

Facico commented 1 year ago

@TY-hue123 Maybe it's a problem with your network? You can try downloading the model file and loading it offline。

Facico commented 1 year ago

@SHITIANYU-hue You can refer to this issue(https://github.com/Facico/Chinese-Vicuna/issues/121#issuecomment-1534012359) Check your GPU and tokenizer. You can try using pure cpu inference to see whether you can produce normal results