search

dusty-nv / jetson-containers

Machine Learning Containers for NVIDIA Jetson and JetPack-L4T
MIT License
1.89k stars 416 forks source link

safetensors' is not a valid JSON file when loading the lava model #485

Open malocker opened 2 months ago

malocker commented 2 months ago

Hello there

am able to load the webui following the instruction from your tutorial : https://www.jetson-ai-lab.com/tutorial_llava.html#1-chat-with-llava-using-text-generation-webui

already downloaded the models but the model not able to load am getting during the load ERROR The model could not be loaded because its checkpoint file in .bin/.pt/.safetensors format could not be located.

when i try to load the model from the webui am getting

OSError: It looks like the config file at '/data/models/text-generation-webui/model.safetensors' is not a valid JSON file.

tried :

  1. download the models and all dependencies manually from huggingface https://huggingface.co/TheBloke/llava-v1.5-13B-GPTQ/tree/main
  2. update torch diffusers

Still not able to load the model in idea how to fix this?

Thank you

dusty-nv commented 2 months ago

@malocker make sure that llava-v1.5-13B-GPTQ is either downloaded under or linked under /data/models/text-generation-webui

Multimodal support in text-generation-webui has also never been very stable, so you may just want to try NanoLLM which I keep working.

malocker commented 2 months ago

@dusty-nv yes downloaded inside text-generation-webui folder but the file name is model.safetensors

i will give a try to NanoLm also

Thank you