Multi-Modal Llava - issue with modle

[brianlmerritt]

When running the multi-modal query here https://www.jetson-ai-lab.com/tutorial_llava.html

./run.sh --workdir=/opt/text-generation-webui $(./autotag text-generation-webui)   python3 server.py --listen     --model-dir /data/models/text-generation-webui     --model TheBloke_llava-v1.5-13B-GPTQ     --multimodal-pipeline llava-v1.5-13b     --loader autogptq     --disable_exllama     --verbose

I get the following error:

Loading TheBloke_llava-v1.5-13B-GPTQ                                                                               
15:32:46-024390 ERROR    The model could not be loaded because its checkpoint file in .bin/.pt/.safetensors format could not be located.   

A potentially unrelated issue on text-generation-webui here

https://github.com/oobabooga/text-generation-webui/discussions/895

Suggests we are running the full rather than 4 bit quantitized model?

[dusty-nv]

Hi @brianlmerritt, can you try checking your files for this model?

$ ls -ll jetson-containers/data/models/text-generation-webui/TheBloke_llava-v1.5-13B-GPTQ/

total 7089852
-rw-r--r-- 1 root root       1905 Dec 20 23:09 config.json
-rw-r--r-- 1 root root        154 Dec 20 23:09 generation_config.json
-rw-r--r-- 1 root root        288 Dec 20 23:09 huggingface-metadata.txt
-rw-r--r-- 1 root root       7020 Dec 20 23:09 LICENSE.txt
-rw-r--r-- 1 root root 7259435072 Dec 20 23:12 model.safetensors
-rw-r--r-- 1 root root        134 Dec 20 23:09 quantize_config.json
-rw-r--r-- 1 root root      19579 Dec 20 23:09 README.md
-rw-r--r-- 1 root root        438 Dec 20 23:09 special_tokens_map.json
-rw-r--r-- 1 root root        748 Dec 20 23:09 tokenizer_config.json
-rw-r--r-- 1 root root     499723 Dec 20 23:09 tokenizer.model
-rw-r--r-- 1 root root       4766 Dec 20 23:09 USE_POLICY.md

It should be stored under your jetson-containers/data directory (this is what /data inside the container gets mounted to) and the .safetensors file should be in there

[brianlmerritt]

It looks like jumping in to the demos without running the build first.