shibing624 / MedicalGPT

MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO。
Apache License 2.0
2.94k stars 451 forks source link

lora模型合并 #355

Open sevenandseven opened 3 months ago

sevenandseven commented 3 months ago

Describe the Question

Please provide a clear and concise description of what the question is.

你好,我使用多个txt文档,lora的方法进行增量预训练,但是合并时出现: return config_cls(**kwargs) TypeError: LoraConfig.init() got an unexpected keyword argument 'layer_replication'

peft是0.9.0 请问这个问题怎么解决?

shibing624 commented 3 months ago

pip install peft==0.7.0

sevenandseven commented 3 months ago

好的,感谢。