root@c8534e683229:/home/MOSS-main# python moss-api.py
Some weights of MossForCausalLM were not initialized from the model checkpoint at ./moss-moon-003-sft-plugin-int8 and are newly initialized because the shapes did not match:
- transformer.h.0.attn.out_proj.qweight: found shape torch.Size([1536, 6144]) in the checkpoint and torch.Size([768, 6144]) in the model instantiated
- transformer.h.0.attn.out_proj.qzeros: found shape torch.Size([48, 1536]) in the checkpoint and torch.Size([48, 768]) in the model instantiated
- transformer.h.0.attn.qkv_proj.qweight: found shape torch.Size([1536, 18432]) in the checkpoint and torch.Size([768, 18432]) in the model instantiated
- transformer.h.0.attn.qkv_proj.qzeros: found shape torch.Size([48, 4608]) in the checkpoint and torch.Size([48, 2304]) in the model instantiated
- transformer.h.0.mlp.fc_in.qweight: found shape torch.Size([1536, 24576]) in the checkpoint and torch.Size([768, 24576]) in the model instantiated
- transformer.h.0.mlp.fc_in.qzeros: found shape torch.Size([48, 6144]) in the checkpoint and torch.Size([48, 3072]) in the model instantiated
- transformer.h.0.mlp.fc_out.qweight: found shape torch.Size([6144, 6144]) in the checkpoint and torch.Size([3072, 6144]) in the model instantiated
- transformer.h.0.mlp.fc_out.qzeros: found shape torch.Size([192, 1536]) in the checkpoint and torch.Size([192, 768]) in the model instantiated
- transformer.h.1.attn.out_proj.qweight: found shape torch.Size([1536, 6144]) in the checkpoint and torch.Size([768, 6144]) in the model instantiated
- transformer.h.1.attn.out_proj.qzeros: found shape torch.Size([48, 1536]) in the checkpoint and torch.Size([48, 768]) in the model instantiated
- transformer.h.1.attn.qkv_proj.qweight: found shape torch.Size([1536, 18432]) in the checkpoint and torch.Size([768, 18432]) in the model instantiated
- transformer.h.1.attn.qkv_proj.qzeros: found shape torch.Size([48, 4608]) in the checkpoint and torch.Size([48, 2304]) in the model instantiated
- transformer.h.1.mlp.fc_in.qweight: found shape torch.Size([1536, 24576]) in the checkpoint and torch.Size([768, 24576]) in the model instantiated
- transformer.h.1.mlp.fc_in.qzeros: found shape torch.Size([48, 6144]) in the checkpoint and torch.Size([48, 3072]) in the model instantiated
- transformer.h.1.mlp.fc_out.qweight: found shape torch.Size([6144, 6144]) in the checkpoint and torch.Size([3072, 6144]) in the model instantiated
- transformer.h.1.mlp.fc_out.qzeros: found shape torch.Size([192, 1536]) in the checkpoint and torch.Size([192, 768]) in the model instantiated
- transformer.h.2.attn.out_proj.qweight: found shape torch.Size([1536, 6144]) in the checkpoint and torch.Size([768, 6144]) in the model instantiated
- transformer.h.2.attn.out_proj.qzeros: found shape torch.Size([48, 1536]) in the checkpoint and torch.Size([48, 768]) in the model instantiated
- transformer.h.2.attn.qkv_proj.qweight: found shape torch.Size([1536, 18432]) in the checkpoint and torch.Size([768, 18432]) in the model instantiated
- transformer.h.2.attn.qkv_proj.qzeros: found shape torch.Size([48, 4608]) in the checkpoint and torch.Size([48, 2304]) in the model instantiated
- transformer.h.2.mlp.fc_in.qweight: found shape torch.Size([1536, 24576]) in the checkpoint and torch.Size([768, 24576]) in the model instantiated
- transformer.h.2.mlp.fc_in.qzeros: found shape torch.Size([48, 6144]) in the checkpoint and torch.Size([48, 3072]) in the model instantiated
- transformer.h.2.mlp.fc_out.qweight: found shape torch.Size([6144, 6144]) in the checkpoint and torch.Size([3072, 6144]) in the model instantiated
- transformer.h.2.mlp.fc_out.qzeros: found shape torch.Size([192, 1536]) in the checkpoint and torch.Size([192, 768]) in the model instantiated
- transformer.h.3.attn.out_proj.qweight: found shape torch.Size([1536, 6144]) in the checkpoint and torch.Size([768, 6144]) in the model instantiated
- transformer.h.3.attn.out_proj.qzeros: found shape torch.Size([48, 1536]) in the checkpoint and torch.Size([48, 768]) in the model instantiated
- transformer.h.3.attn.qkv_proj.qweight: found shape torch.Size([1536, 18432]) in the checkpoint and torch.Size([768, 18432]) in the model instantiated
- transformer.h.3.attn.qkv_proj.qzeros: found shape torch.Size([48, 4608]) in the checkpoint and torch.Size([48, 2304]) in the model instantiated
使用 moss-moon-003-sft-plugin-int8 加载模型提示如下错误，接口返回的乱码