ymcui / Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki
Apache License 2.0
18.23k stars 1.86k forks source link

预训练从1000 steps断点处加载模型参数报错 #814

Closed zzzhaoguziji closed 1 year ago

zzzhaoguziji commented 1 year ago

提交前必须检查以下项目

问题类型

模型训练与精调

基础模型

LLaMA-7B

操作系统

Linux

详细描述问题

# 请在此处粘贴运行代码(如没有可删除该代码块)

涉及代码主要为 run_clm_pt_with_peft.py 中的内容。 注释 old_state_dict = model.state_dict model.statedict = ( lambda self, *, **: get_peft_model_state_dict(self, old_state_dict()) ).get__(model, type(model)) 后仍然不行。

依赖情况(代码类问题务必提供)

# 请在此处粘贴依赖情况

运行日志或截图

# 请在此处粘贴运行日志
llama2报错
Qmymy commented 1 year ago

同样的问题

Qmymy commented 1 year ago

已经找到了解决方法https://github.com/ymcui/Chinese-LLaMA-Alpaca/issues/464#issuecomment-1568328617

zzzhaoguziji commented 1 year ago

已经找到了解决方法#464 (comment)

感谢

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your consideration.

github-actions[bot] commented 1 year ago

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.