下载的checkpoint无法正常加载：Missing key(s) in state_dict: "transformer.position_embeddings.weight".

[makeme-zgz]

RuntimeError: Error(s) in loading state_dict for GPT2Model: Missing key(s) in state_dict: "transformer.position_embeddings.weight". Unexpected key(s) in state_dict: "transformer.r_w_bias", "transformer.r_r_bias", "transformer.position_embeddings.inv_freq", "transformer.layers.0.attention.relative.weight", "transformer.layers.0.attention.relative.bias", "transformer.layers.1.attention.relative.weight", "transformer.layers.1.attention.relative.bias", "transformer.layers.2.attention.relative.weight", "transformer.layers.2.attention.relative.bias", "transformer.layers.3.attention.relative.weight", "transformer.layers.3.attention.relative.bias", "transformer.layers.4.attention.relative.weight", "transformer.layers.4.attention.relative.bias", "transformer.layers.5.attention.relative.weight", "transformer.layers.5.attention.relative.bias", "transformer.layers.6.attention.relative.weight", "transformer.layers.6.attention.relative.bias", "transformer.layers.7.attention.relative.weight", "transformer.layers.7.attention.relative.bias", "transformer.layers.8.attention.relative.weight", "transformer.layers.8.attention.relative.bias", "transformer.layers.9.attention.relative.weight", "transformer.layers.9.attention.relative.bias", "transformer.layers.10.attention.relative.weight", "transformer.layers.10.attention.relative.bias", "transformer.layers.11.attention.relative.weight", "transformer.layers.11.attention.relative.bias", "transformer.layers.12.attention.relative.weight", "transformer.layers.12.attention.relative.bias", "transformer.layers.13.attention.relative.weight", "transformer.layers.13.attention.relative.bias", "transformer.layers.14.attention.relative.weight", "transformer.layers.14.attention.relative.bias", "transformer.layers.15.attention.relative.weight", "transformer.layers.15.attention.relative.bias", "transformer.layers.16.attention.relative.weight", "transformer.layers.16.attention.relative.bias", "transformer.layers.17.attention.relative.weight", "transformer.layers.17.attention.relative.bias", "transformer.layers.18.attention.relative.weight", "transformer.layers.18.attention.relative.bias", "transformer.layers.19.attention.relative.weight", "transformer.layers.19.attention.relative.bias", "transformer.layers.20.attention.relative.weight", "transformer.layers.20.attention.relative.bias", "transformer.layers.21.attention.relative.weight", "transformer.layers.21.attention.relative.bias", "transformer.layers.22.attention.relative.weight", "transformer.layers.22.attention.relative.bias", "transformer.layers.23.attention.relative.weight", "transformer.layers.23.attention.relative.bias", "transformer.layers.24.attention.relative.weight", "transformer.layers.24.attention.relative.bias", "transformer.layers.25.attention.relative.weight", "transformer.layers.25.attention.relative.bias", "transformer.layers.26.attention.relative.weight", "transformer.layers.26.attention.relative.bias", "transformer.layers.27.attention.relative.weight", "transformer.layers.27.attention.relative.bias", "transformer.layers.28.attention.relative.weight", "transformer.layers.28.attention.relative.bias", "transformer.layers.29.attention.relative.weight", "transformer.layers.29.attention.relative.bias", "transformer.layers.30.attention.relative.weight", "transformer.layers.30.attention.relative.bias", "transformer.layers.31.attention.relative.weight", "transformer.layers.31.attention.relative.bias".

[duzx16]

你好，我没法复现您描述的错误，您是否对代码做出了改动？可以重新git clone再试一下

[makeme-zgz]

重新拉了一遍代码之后又可以了， 貌似是small model的script不行。