LLaVA-VL / LLaVA-NeXT

Apache License 2.0
2.79k stars 225 forks source link

Running LLaVA_OneVision_Tutorials.ipynb reports an ValueError: Unknown vision tower: google/siglip-so400m-patch14-384 #216

Open MXC66ai opened 1 month ago

MXC66ai commented 1 month ago

LLaVA_OneVision_Tutorials_ValueError

ValueError Traceback (most recent call last)

in () 3 4 # Load the pre-trained model without passing the vision_tower parameter ----> 5 tokenizer, model, image_processor, max_length = load_pretrained_model( 6 pretrained, 7 None, 5 frames /usr/local/lib/python3.10/dist-packages/llava/model/multimodal_encoder/builder.py in build_vision_tower(vision_tower_cfg, **kwargs) 13 return CLIPVisionTower(vision_tower, args=vision_tower_cfg, **kwargs) 14 ---> 15 raise ValueError(f'Unknown vision tower: {vision_tower}') ValueError: Unknown vision tower: google/siglip-so400m-patch14-384 Is there any way to solve the Unknown vision tower problem?
mylesgoose commented 1 month ago

`import os from transformers import AutoModel, logging

Suppress warnings (optional)

logging.set_verbosity_error()

Model identifier

model_name = "google/siglip-so400m-patch14-384"

Check if the model is already cached

cache_dir = "~/.cache/huggingface/hub" # Default cache directory model_path = os.path.join(cache_dir, "models--google--siglip-so400m-patch14-384") if os.path.exists(model_path): print(f"Model '{model_name}' is already cached at: {model_path}") else: print(f"Model '{model_name}' is not cached. Downloading...")

# Download the model
try:
    _ = AutoModel.from_pretrained(model_name)
    print(f"Model '{model_name}' downloaded successfully!")
except Exception as e:
    print(f"Error downloading model '{model_name}': {e}")`