search

haotian-liu / LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
https://llava.hliu.cc
Apache License 2.0
20.03k stars 2.21k forks source link

[Usage] KeyError: 'llava' #386

Closed pipixiaqishi1 closed 1 year ago

pipixiaqishi1 commented 1 year ago

Issue:

Try to download this https://huggingface.co/liuhaotian/llava-llama-2-13b-chat-lightning-preview, using:

# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("liuhaotian/llava-llama-2-13b-chat-lightning-preview")

face the problem: File ".py", line 3, in model = AutoModelForCausalLM.from_pretrained("liuhaotian/llava-llama-2-13b-chat-lightning-preview") File "/envs/llava/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 444, in from_pretrained config, kwargs = AutoConfig.from_pretrained( File "/envs/llava/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 940, in from_pretrained config_class = CONFIG_MAPPING[config_dict["model_type"]] File "/envs/llava/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 655, in getitem raise KeyError(key) KeyError: 'llava'

Reinstall transformers but it didn't work...

JadarTheObscurity commented 1 year ago

Hey @pipixiaqishi1 I meet the same problem, do you have the solution?

JadarTheObscurity commented 1 year ago

Solved with following python script

from transformers import AutoConfig, LlamaConfig 
from llava.model.language_model.llava_llama import LlavaLlamaForCausalLM
class LlavaConfig(LlamaConfig):
    model_type = "llava"
AutoConfig.register("llava", LlavaConfig)
model = LlavaLlamaForCausalLM.from_pretrained("liuhaotian/llava-llama-2-13b-chat-lightning-preview")
pipixiaqishi1 commented 1 year ago

Solved with following python script

from transformers import AutoConfig, LlamaConfig 
from llava.model.language_model.llava_llama import LlavaLlamaForCausalLM
class LlavaConfig(LlamaConfig):
    model_type = "llava"
AutoConfig.register("llava", LlavaConfig)
model = LlavaLlamaForCausalLM.from_pretrained("liuhaotian/llava-llama-2-13b-chat-lightning-preview")

Yes! I also found this code at https://github.com/haotian-liu/LLaVA/blob/237165b0166ad3fdc4fb0a7d881c857755d2404f/llava/model/language_model/llava_llama.py#L30-L42 and followed the way to load the pretrained model. Thanks for sharing that!