InternLM / InternLM-XComposer

InternLM-XComposer2 is a groundbreaking vision-language large model (VLLM) excelling in free-form text-image composition and comprehension.
1.91k stars 120 forks source link

Some errors:ValueError: Tokenizer class InternLMXComposerTokenizer does not exist or is not currently imported. #308

Open Shuweis opened 1 month ago

Shuweis commented 1 month ago

I run the quick inference demo of the github repo and follow the install pipeline without any other operation. But I got this error. Traceback (most recent call last): File "/home/ssw/DemoFusion/base_models/InternLM-XComposer/projects/ShareGPT4V/demo_share4v.py", line 24, in eval_model(args) File "/home/ssw/DemoFusion/base_models/InternLM-XComposer/projects/ShareGPT4V/share4v/eval/run_share4v.py", line 31, in eval_model tokenizer, model, image_processor, context_len = load_pretrained_model( File "/home/ssw/DemoFusion/base_models/InternLM-XComposer/projects/ShareGPT4V/share4v/model/builder.py", line 114, in load_pretrained_model tokenizer = AutoTokenizer.from_pretrained( File "/home/ssw/anaconda3/envs/share4v/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 699, in from_pretrained raise ValueError( ValueError: Tokenizer class InternLMXComposerTokenizer does not exist or is not currently imported.

Hope for your reply!

Shuweis commented 1 month ago

 怀疑可能是sharecaptioner的inference环境和sharegpt4v-7B不一样?我的后者模型是可以正常运行的,但是前者不行