Hello, thank you for your work
When I run bash launch_belle_cmd.sh, I got this error
Is this model: BelleGroup--BELLE-LLaMA-7B-2M-enc out of version?
Or you could zip up and send me a copy of your mock-up BelleGroup--BELLE-LLaMA-7B-2M-enc
Traceback (most recent call last):
File "/home/Hongwei/disk_hdd/Tiantian/LLM_Memory/MemoryBank-SiliconFriend-main/SiliconFriend-ChatGLM-BELLE/cli_demo.py", line 86, in
tokenizer, model= load_belle_tokenizer_and_model(model_args.base_model,model_args.adapter_model)
File "/home/Hongwei/disk_hdd/Tiantian/LLM_Memory/MemoryBank-SiliconFriend-main/SiliconFriend-ChatGLM-BELLE/../utils/model_utils.py", line 67, in load_belle_tokenizer_and_model
tokenizer = LlamaTokenizer.from_pretrained(base_model)
File "/home/Hongwei/anaconda3/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "/home/Hongwei/anaconda3/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1876, in _from_pretrained
init_kwargs = json.load(tokenizer_config_handle)
File "/home/Hongwei/anaconda3/lib/python3.9/json/init.py", line 293, in load
return loads(fp.read(),
File "/home/Hongwei/anaconda3/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa6 in position 0: invalid start byte
Hello, thank you for your work When I run bash launch_belle_cmd.sh, I got this error Is this model: BelleGroup--BELLE-LLaMA-7B-2M-enc out of version? Or you could zip up and send me a copy of your mock-up BelleGroup--BELLE-LLaMA-7B-2M-enc
Traceback (most recent call last): File "/home/Hongwei/disk_hdd/Tiantian/LLM_Memory/MemoryBank-SiliconFriend-main/SiliconFriend-ChatGLM-BELLE/cli_demo.py", line 86, in
tokenizer, model= load_belle_tokenizer_and_model(model_args.base_model,model_args.adapter_model)
File "/home/Hongwei/disk_hdd/Tiantian/LLM_Memory/MemoryBank-SiliconFriend-main/SiliconFriend-ChatGLM-BELLE/../utils/model_utils.py", line 67, in load_belle_tokenizer_and_model
tokenizer = LlamaTokenizer.from_pretrained(base_model)
File "/home/Hongwei/anaconda3/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "/home/Hongwei/anaconda3/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1876, in _from_pretrained
init_kwargs = json.load(tokenizer_config_handle)
File "/home/Hongwei/anaconda3/lib/python3.9/json/init.py", line 293, in load
return loads(fp.read(),
File "/home/Hongwei/anaconda3/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa6 in position 0: invalid start byte