Re. Model path does not exist: /root/.cache/py-llm-core/models/BakLLaVA-1-Q4_K_M.gguf

[andysingal]

While working on Baklaava Model

!wget https://huggingface.co/advanced-stack/bakllava-mistral-v1-gguf/resolve/main/BakLLaVA-1-Q4_K_M.gguf
!wget https://huggingface.co/advanced-stack/bakllava-mistral-v1-gguf/resolve/main/BakLLaVA-1-clip-model.gguf

Download:

!pip install py-llm-core -q
!pip install --upgrade --verbose --force-reinstall --no-cache-dir llama-cpp-python -q

from llm_core.llm import LLaVACPPModel

model = "BakLLaVA-1-Q4_K_M.gguf"

llm = LLaVACPPModel(
    name=model,
    llama_cpp_kwargs={
        "logits_all": True,
        "n_ctx": 8000,
        "verbose": False,
        "n_gpu_layers": 100,  #: Set to 0 if you don't have a GPU
        "n_threads": 1,       #: Set to the number of available CPU cores
        "clip_model_path": "BakLLaVA-1-clip-model.gguf"
    }
)

llm.load_model()

history = [
    {
        'role': 'user',
        'content': [
            {'type': 'image_url', 'image_url': 'https://advanced-stack.com/assets/img/mappemonde.jpg'}
        ]
    }
]

response = llm.ask('Describe the image as accurately as possible', history=history)

print(response.choices[0].message.content)

ERROR:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[7], line 17
      3 model = "BakLLaVA-1-Q4_K_M.gguf"
      5 llm = LLaVACPPModel(
      6     name=model,
      7     llama_cpp_kwargs={
   (...)
     14     }
     15 )
---> 17 llm.load_model()
     19 history = [
     20     {
     21         'role': 'user',
   (...)
     25     }
     26 ]
     28 response = llm.ask('Describe the image as accurately as possible', history=history)

File /usr/local/lib/python3.10/dist-packages/llm_core/llm/llama_cpp_compatible.py:129, in LLaVACPPModel.load_model(self)
    124 chat_format = "llava-1-5"
    125 chat_handler = llama_cpp.llama_chat_format.Llava15ChatHandler(
    126     clip_model_path=self.llama_cpp_kwargs["clip_model_path"]
    127 )
--> 129 self.model = llama_cpp.Llama(
    130     model_path,
    131     chat_format=chat_format,
    132     chat_handler=chat_handler,
    133     **self.llama_cpp_kwargs,
    134 )

File /usr/local/lib/python3.10/dist-packages/llama_cpp/llama.py:887, in Llama.__init__(self, model_path, n_gpu_layers, main_gpu, tensor_split, vocab_only, use_mmap, use_mlock, seed, n_ctx, n_batch, n_threads, n_threads_batch, rope_scaling_type, rope_freq_base, rope_freq_scale, yarn_ext_factor, yarn_attn_factor, yarn_beta_fast, yarn_beta_slow, yarn_orig_ctx, mul_mat_q, f16_kv, logits_all, embedding, last_n_tokens_size, lora_base, lora_scale, lora_path, numa, chat_format, chat_handler, verbose, **kwargs)
    884 self.lora_path = lora_path
    886 if not os.path.exists(model_path):
--> 887     raise ValueError(f"Model path does not exist: {model_path}")
    889 self._model = _LlamaModel(
    890     path_model=self.model_path, params=self.model_params, verbose=self.verbose
    891 )
    893 self._ctx = _LlamaContext(
    894     model=self._model,
    895     params=self.context_params,
    896     verbose=self.verbose,
    897 )

ValueError: Model path does not exist: /root/.cache/py-llm-core/models/BakLLaVA-1-Q4_K_M.gguf

[paschembri]

You need to ensure your models are present in /root/.cache/py-llm-core/models/

You can always change the directory using the MODELS_CACHE_DIR environment variable

[andysingal]

Thanks , her is follow up question :

• When to add root directory when models are available in /workspace/models

I want to run bash script on multiple images but Jupyter notebook dies on RTX 4090 .

‘’’

Where i need help::::

%%bash

Define the directory containing the images

IMG_DIR=~/Desktop/Papers/LLaVA/

Loop through each image in the directory

for img in "${IMG_DIR}"*.jpg; do

Extract the base name of the image without extension

base_name=$(basename "$img" .jpg)

# Define the output file name based on the image name
output_file="${IMG_DIR}${base_name}.txt"

# Execute the command and save the output to the defined output file
/Users/rlm/Desktop/Code/llama.cpp/bin/llava -m

../models/llava-7b/ggml-model-q5_k.gguf --mmproj ../models/llava-7b/mmproj-model-f16.gguf --temp 0.1 -p "Describe the image in detail. Be specific about graphs, such as bar plots." --image "$img" > "$output_file"

done

‘’’

On Tue, Nov 28, 2023 at 01:27 Pierre Alexandre SCHEMBRI < @.***> wrote:

You need to ensure your models are present in /root/.cache/py-llm-core/models/

You can always change the directory using the MODELS_CACHE_DIR environment variable

MODELS_CACHE_DIR = config( "MODELS_CACHE_DIR", default=os.path.expanduser("~/.cache/py-llm-core/models"), )

— Reply to this email directly, view it on GitHub https://github.com/advanced-stack/py-llm-core/issues/5#issuecomment-1829194950, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE4LJNOT4CGWLEJ3XPIMPQ3YGV74RAVCNFSM6AAAAAA7X6ZJBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRZGE4TIOJVGA . You are receiving this because you authored the thread.Message ID: @.***>

[paschembri]

Closing stale issue.