AttributeError: 'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'. Did you mean: '_tokenize'?

THUDM / VisualGLM-6B

Chinese and English multimodal conversational language model | 多模态中英双语对话语言模型

Apache License 2.0

4.08k stars 416 forks source link

AttributeError: 'ChatGLMTokenizer' object has no attribute 'sp_tokenizer'. Did you mean: '_tokenize'? #323

Open MetaDreamer3d opened 10 months ago

MetaDreamer3d commented 10 months ago

hi , can you help me?

KinokoY commented 10 months ago

try pip install transformers==4.33.0

floatingwh commented 5 months ago

meet the same problem. How to fix it ?try pip install transformers==4.33.0 can not solve it

fengpeiyuan commented 4 months ago

moving line "self.sp_tokenizer = SPTokenizer(vocab_file, num_image_tokens=num_image_tokens) " before "super().init(" at "init" of class ChatGLMTokenizer can solve this issue.