I followed the instruction to create a new conda environment, and following the inference script to quick load the model.
(mplug_owl2) alex@HP-machine:~/mPLUG-Owl/mPLUG-Owl2$ python test_inference_mplugowl.py [2024-02-10 16:54:58,039] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) Traceback (most recent call last): File "/home/alex/mPLUG-Owl/mPLUG-Owl2/test_inference_mplugowl.py", line 15, in <module> tokenizer, model, image_processor, context_len = load_pretrained_model(model_path, None, model_name, load_8bit=False, load_4bit=False, device="cuda") File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/builder.py", line 117, in load_pretrained_model model = AutoModelForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, **kwargs) File "/home/alex/mambaforge-pypy3/envs/mplug_owl2/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained return model_class.from_pretrained( File "/home/alex/mambaforge-pypy3/envs/mplug_owl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2700, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/modeling_mplug_owl2.py", line 218, in __init__ self.model = MPLUGOwl2LlamaModel(config) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/modeling_mplug_owl2.py", line 205, in __init__ super(MPLUGOwl2LlamaModel, self).__init__(config) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/modeling_mplug_owl2.py", line 36, in __init__ self.vision_model = MplugOwlVisionModel( File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/visual_encoder.py", line 403, in __init__ self.embeddings = MplugOwlVisionEmbeddings(config) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/visual_encoder.py", line 105, in __init__ if self.cls_token: RuntimeError: Boolean value of Tensor with more than one value is ambiguous
I followed the instruction to create a new conda environment, and following the inference script to quick load the model.
(mplug_owl2) alex@HP-machine:~/mPLUG-Owl/mPLUG-Owl2$ python test_inference_mplugowl.py [2024-02-10 16:54:58,039] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) Traceback (most recent call last): File "/home/alex/mPLUG-Owl/mPLUG-Owl2/test_inference_mplugowl.py", line 15, in <module> tokenizer, model, image_processor, context_len = load_pretrained_model(model_path, None, model_name, load_8bit=False, load_4bit=False, device="cuda") File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/builder.py", line 117, in load_pretrained_model model = AutoModelForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, **kwargs) File "/home/alex/mambaforge-pypy3/envs/mplug_owl2/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 493, in from_pretrained return model_class.from_pretrained( File "/home/alex/mambaforge-pypy3/envs/mplug_owl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2700, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/modeling_mplug_owl2.py", line 218, in __init__ self.model = MPLUGOwl2LlamaModel(config) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/modeling_mplug_owl2.py", line 205, in __init__ super(MPLUGOwl2LlamaModel, self).__init__(config) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/modeling_mplug_owl2.py", line 36, in __init__ self.vision_model = MplugOwlVisionModel( File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/visual_encoder.py", line 403, in __init__ self.embeddings = MplugOwlVisionEmbeddings(config) File "/home/alex/mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/visual_encoder.py", line 105, in __init__ if self.cls_token: RuntimeError: Boolean value of Tensor with more than one value is ambiguous
Here is the script