ValueError: 130004 is not in list

[PappeSister]

用qlora微调报错：ValueError: 130004 is not in list，报错信息： context_lengths = [seq.tolist().index(self.bos_token_id) for seq in input_ids] ValueError: 130004 is not in list [2023-08-09 15:02:18,539] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 23563 [2023-08-09 15:02:18,767] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 23564 [2023-08-09 15:02:18,789] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 23565 [2023-08-09 15:02:18,811] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 23566 [2023-08-09 15:02:18,811] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 23567

打印input_ids和tokenizer.bos_token_id： ++++++++++++++++++++ 150004 ++++++++++ **** [20046, 22265, 20033, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20098, 22265, 20033, 84286, 20012, 94366, 85842, 101370, 83884, 86346, 84308, 20031, 20004, 87342, 20012, 20005, 94366, 85842, 101370, 83829, 97894, 101675, 83823, 150001, 150004] ** ++++++++++++++++++++ 150004 ++++++++++ **** [20046, 22265, 20033, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20003, 20098, 22265, 20033, 84286, 20012, 94366, 85842, 101370, 83884, 86346, 84308, 20031, 20004, 87342, 20012, 20005, 94366, 85842, 101370, 83829, 92401, 86429, 83826, 101359, 83823, 150001, 150004] **

[1049451037]

应该是你的tokenizer没有更新，请从huggingface下载最新的chatglm的tokenizer：https://huggingface.co/THUDM/chatglm-6b/tree/main