Open HeroSong666 opened 1 year ago
运行下面单元格时： torch.set_default_tensor_type(torch.cuda.HalfTensor)
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained( "THUDM/chatglm-6b", trust_remote_code=True, device_map=DeviceMap("ChatGLM").get() ) 出现报错： AttributeError Traceback (most recent call last) in <cell line: 3>() 1 torch.set_default_tensor_type(torch.cuda.HalfTensor) 2 ----> 3 tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) 4 5 model = AutoModel.from_pretrained(
7 frames ~/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/8b7d33596d18c5e83e2da052d05ca4db02e60620/tokenization_chatglm.py in vocab_size(self) 242 def vocab_size(self): 243 """ Returns vocab size """ --> 244 return self.sp_tokenizer.num_tokens 245 246 def get_vocab(self):
AttributeError: 'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'
运行下面单元格时： torch.set_default_tensor_type(torch.cuda.HalfTensor)
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained( "THUDM/chatglm-6b", trust_remote_code=True, device_map=DeviceMap("ChatGLM").get() ) 出现报错： AttributeError Traceback (most recent call last)
7 frames ~/.cache/huggingface/modules/transformers_modules/THUDM/chatglm-6b/8b7d33596d18c5e83e2da052d05ca4db02e60620/tokenization_chatglm.py in vocab_size(self) 242 def vocab_size(self): 243 """ Returns vocab size """ --> 244 return self.sp_tokenizer.num_tokens 245 246 def get_vocab(self):
AttributeError: 'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'