QLORA微调alpaca_data.json报错 'padding_value' (position 3) must be float, not NoneType

[kanxueli]

@jianzhnie 感谢你的工作！我在复现代码时遇到一个问题，想请教您一下: 我调用scripts/finetune_llama_7b_alpaca_zh.sh，微调alpaca_data.json数据集时。出现以下报错，请问是什么问题呀?

...... File "/home/likanxue1/.custom/cuda11.3.1-cudnn8-devel-ubuntu20.04-py38-jupyter/envs/tuning-llm/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch return self.collate_fn(data) File "/home/likanxue1/model_inference/transformers/src/transformers/trainer_utils.py", line 772, in call return self.data_collator(features) File "/home/likanxue1/model_inference/Efficient-Tuning-LLMs/chatllms/data/sft_dataset.py", line 214, in call input_ids = pad_sequence(input_ids, File "/home/likanxue1/.custom/cuda11.3.1-cudnn8-devel-ubuntu20.04-py38-jupyter/envs/tuning-llm/lib/python3.10/site-packages/torch/nn/utils/rnn.py", line 399, in pad_sequence return torch._C._nn.pad_sequence(sequences, batch_first, padding_value) TypeError: pad_sequence(): argument 'padding_value' (position 3) must be float, not NoneType

我在/data/alpaca_zh_pcyn.yaml中只有以下几行信息： alpaca: hf_hub_url: '' local_path: /home/likanxue1/Dataset/alpaca_data.json dataset_format: alpaca multi_turn: False

期待你的及时回复，多有打扰，请海涵！

[jianzhnie]

似乎是因为你的 tokenizer 没有pad_token_id

[kanxueli]

是的，谢谢你的答复！已经解决的了，在special_tokens_map.json中添加“pad_token”: ""/“pad_token”: ""就可以了