[Question] 模型参数问题

[L-hongbin]

Required prerequisites

• [X] I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
• [X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• [X] Consider asking first in a Discussion.

Questions

请问lm_head.weight 和 model.embed_tokens.weight 的值为什么是不一样的呢，一般在预训练中会复用model.embed_tokens.weight 来映射回词表，所以感觉一般是一样的。这里是在预训练的时候把 model.embed_tokens.weight 和 lm_head.weight 分开了吗？

Checklist

• [X] I have provided all relevant and necessary information above.
• [X] I have chosen a suitable title for this issue.