search

project-baize / baize-chatbot

Let ChatGPT teach your own chatbot in hours with a single GPU!
https://arxiv.org/abs/2304.01196
GNU General Public License v3.0
3.15k stars 275 forks source link

train my own data, answers are not very accurate. #36

Open yfq512 opened 1 year ago

yfq512 commented 1 year ago

I collect some chinese data about "中国云南" like this: 0417-2 And train follow the readme base on Baize-7B, cost 48 hours, get checkpoints finally. when I use this checkpoints to run app.py. The AI can speak Chinese, but sometimes it mixes English and Russian, and the answers are not very accurate. 0417

Can you help me analyze the cause? Is it a lack of training data? Or the base model is an English model? Or something else? thanks

JetRunner commented 1 year ago

Are you using LLaMA as the foundation model? If so, as LLaMA has no Chinese pretraining data, it's not very surprising that the outcome isn't very good. BLOOMZ may be a better foundation model.