原封不动下载的代码和数据，在已经成功运行官方版本的环境里出错

[xianglei3]

File "/software/anaconda3/envs/chatglm/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 3476, in decode return self._decode( File "/root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/9324de70a93207c9a310cf99d5d6261791489691/tokenization_chatglm.py", line 262, in _decode return self.sp_tokenizer.decode(token_ids) File "/root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/9324de70a93207c9a310cf99d5d6261791489691/tokenization_chatglm.py", line 117, in decode text = self._get_text_tokenizer().decode(ids) File "/root/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/9324de70a93207c9a310cf99d5d6261791489691/tokenization_chatglm.py", line 29, in decode return self.sp.DecodeIds(ids) File "/software/anaconda3/envs/chatglm/lib/python3.8/site-packages/sentencepiece/init.py", line 837, in DecodeIds return self.Decode(input=input, out_type=out_type, **kwargs) File "/software/anaconda3/envs/chatglm/lib/python3.8/site-packages/sentencepiece/init.py", line 780, in Decode return self._DecodeIds(input) File "/software/anaconda3/envs/chatglm/lib/python3.8/site-packages/sentencepiece/init.py", line 337, in _DecodeIds return _sentencepiece.SentencePieceProcessor__DecodeIds(self, ids) IndexError: Out of range: piece id is out of range. (chatglm) root@ls-System-Product-Name:/home/xl/Chatglm6b_ModelParallel_ptuning# ^C

[djyaya]

同问

[yuanzhoulvpi2017]

这个bug已经修复了。可以查看链接🔗https://github.com/yuanzhoulvpi2017/zero_nlp/tree/main/simple_thu_chatglm6b#04-15-%E7%89%88%E6%9C%AC