baichuan-inc / Baichuan-7B

A large-scale 7B pretraining language model developed by BaiChuan-Inc.
https://huggingface.co/baichuan-inc/baichuan-7B
Apache License 2.0
5.67k stars 506 forks source link

启动train_sft.py时,报异常信息Target modules ['q_proj', 'v_proj'] not found [Question] #86

Closed grantchenhuarong closed 1 year ago

grantchenhuarong commented 1 year ago

Required prerequisites

Questions

启动train_sft.py时,提示 “ValueError: Target modules ['q_proj', 'v_proj'] not found in the base model. Please check the target modules and try again.”

请问是模型下载的配置文件不完整么?

Checklist

grantchenhuarong commented 1 year ago

启动命令为:CUDA_VISIBLE_DEVICES=2,3 python src/train_sft.py \ --model_name_or_path /data/ftp/models/baichuan \ --do_train \ --dataset guanaco_belle_merge \ --finetuning_type lora \ --output_dir outs/baichuan_sft \ --overwrite_cache \ --per_device_train_batch_size 4 \ --gradient_accumulation_steps 4 \ --lr_scheduler_type cosine \ --logging_steps 10 \ --save_steps 1000 \ --learning_rate 5e-5 \ --num_train_epochs 3.0 \ --plot_loss \ --fp16

grantchenhuarong commented 1 year ago

好像发错项目了。。。