[Question] 两个小坑：没有pad_token && 把没必要的buffer保存下来了

[zlkqz]

Required prerequisites

• [X] I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
• [X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• [x] Consider asking first in a Discussion.

Questions

1. 在训练时，需要padding，但是pad_token_id并没有跟着tokenizer一起保存下来

   >>> tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/baichuan-7B")
   >>> print(tokenizer.pad_token_id, tokenizer.pad_token)
   Output: None, None

   如果不指定pad_token_id会触发Cuda Error越界问题或者pad_token not found

2. 在 https://github.com/baichuan-inc/baichuan-7B/blob/main/models/modeling_baichuan.py 里96行可以看到，注册到buffer的inv_freq跟着state_dict()一起保存下来了，但是这个在推理时完全没用！如果用model.named_parameters()保存模型参数，会报错没有找到model.model.layers.31.self_attn.rotary_emb.inv_freq

Checklist

• [X] I have provided all relevant and necessary information above.
• [X] I have chosen a suitable title for this issue.

[zlkqz]

1. 第一个问题：将pad_token_id设为0

   >>> tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/baichuan-7B")
   >>> tokenizer.pad_token_id = 0

2. 第二个问题：将该行代码加一个persistent=False参数

   >>> self.register_buffer("inv_freq", inv_freq, persistent=False)

[TangoW]

1. 第一个问题：将pad_token_id设为0

>>> tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/baichuan-7B")
>>> tokenizer.pad_token_id = 0

2. 第二个问题：将该行代码加一个persistent=False参数

>>> self.register_buffer("inv_freq", inv_freq, persistent=False)

也可以在special_tokens_map.json中添加一个pad_token，内容直接复制unk_token，即：

"pad_token": {
    "content": "<unk>",
    "lstrip": false,
    "normalized": true,
    "rstrip": false,
    "single_word": false
  }

[yqli2420]

1. 第一个问题：将pad_token_id设为0

>>> tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/baichuan-7B")
>>> tokenizer.pad_token_id = 0

2. 第二个问题：将该行代码加一个persistent=False参数

>>> self.register_buffer("inv_freq", inv_freq, persistent=False)

改成tokenizer.pad_token_id = 0微调实验，跑着跑着还会出现Cuda Error越界问题