Command - local-llm run meta-llama/Llama-3.2-11B-Vision 8000 --verbose
Issues - Not able to download and run the LLM model
Traceback (most recent call last):
File "/home/ubuntu/.local/bin/uvicorn", line 8, in <module>
sys.exit(main())
File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/ubuntu/.local/lib/python3.10/site-packages/uvicorn/main.py", line 410, in main
run(
File "/home/ubuntu/.local/lib/python3.10/site-packages/uvicorn/main.py", line 577, in run
server.run()
File "/home/ubuntu/.local/lib/python3.10/site-packages/uvicorn/server.py", line 65, in run
return asyncio.run(self.serve(sockets=sockets))
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/ubuntu/.local/lib/python3.10/site-packages/uvicorn/server.py", line 69, in serve
await self._serve(sockets)
File "/home/ubuntu/.local/lib/python3.10/site-packages/uvicorn/server.py", line 76, in _serve
config.load()
File "/home/ubuntu/.local/lib/python3.10/site-packages/uvicorn/config.py", line 440, in load
self.loaded_app = self.loaded_app()
File "/home/ubuntu/.local/lib/python3.10/site-packages/llama_cpp/server/app.py", line 150, in create_app
set_llama_proxy(model_settings=model_settings)
File "/home/ubuntu/.local/lib/python3.10/site-packages/llama_cpp/server/app.py", line 70, in set_llama_proxy
_llama_proxy = LlamaProxy(models=model_settings)
File "/home/ubuntu/.local/lib/python3.10/site-packages/llama_cpp/server/model.py", line 31, in __init__
self._current_model = self.load_llama_from_model_settings(
File "/home/ubuntu/.local/lib/python3.10/site-packages/llama_cpp/server/model.py", line 236, in load_llama_from_model_settings
_model = create_fn(
File "/home/ubuntu/.local/lib/python3.10/site-packages/llama_cpp/llama.py", line 369, in __init__
internals.LlamaModel(
File "/home/ubuntu/.local/lib/python3.10/site-packages/llama_cpp/_internals.py", line 56, in __init__
raise ValueError(f"Failed to load model from file: {path_model}")
ValueError: Failed to load model from file: /home/ubuntu/.cache/huggingface/hub/models--meta-llama--Llama-3.2-11B-Vision/refs/main
Error starting LLM, run with --verbose for more
Command -
local-llm run meta-llama/Llama-3.2-11B-Vision 8000 --verbose
Issues - Not able to download and run the LLM model