ssbuild / chatglm_finetuning

chatglm 6b finetuning and alpaca finetuning
1.53k stars 175 forks source link

您好,非常感谢您的工作。我进行全参数微调后,调用infer_finetuning.py的时候,出现Missing key(s) in state_dict: "_TransformerLightningModule__backbone.transformer.lm_head.weight".请问您有遇到过这个问题吗 #252

Closed Xuan-ZW closed 1 year ago

ssbuild commented 1 year ago

https://github.com/ssbuild/chatglm_finetuning/blob/089404c0ffd59ea5c9e2d889ccb4c48f866c867c/infer_finetuning.py#L40 lm_head 是共享权重 ,修改 strict = False 即可

MrTFK commented 1 year ago

我这边修改strict=False后输出的结果是很差的,基本等同乱码,是不是不太能这么搞,感觉就像参数错乱了

比如: input 影响黄金价格的因素有哪些? output 等等。等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,等等,474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747474747