search

evilsocket / cake

Distributed LLM and StableDiffusion inference for mobile, desktop and server.
Other
2.57k stars 135 forks source link

May I ask why I am unable to download the model and use the product through Huggingface #23

Closed Caesarleee closed 1 week ago

Caesarleee commented 2 months ago

root@llama01:/www/cake# /www/cake/target/release/cake-cli --model /www/llama --mode worker --name linux_server_1 --address 0.0.0.0:9527 --topology /www/cake/topology.yml [2024-08-08T16:11:12Z INFO ] [Worker] dtype=F16 device=Cpu mem=5.3 MiB [2024-08-08T16:11:12Z INFO ] loading configuration from /www/llama/config.json [2024-08-08T16:11:12Z INFO ] loading topology from /www/cake/topology.yml [2024-08-08T16:11:12Z INFO ] loading tensors in /www/llama/model.safetensors.index.json [2024-08-08T16:11:12Z INFO ] loading tensors from /www/llama/model.safetensors.index.json ... [2024-08-08T16:11:12Z INFO ] loading model-00002-of-00004.safetensors ... Error: cannot find tensor model-00002-of-00004.safetensors.self_attn.q_proj.weight

Caesarleee commented 2 months ago

Error: cannot find tensor model-00002-of-00004.safetensors.self_attn.q_proj.weight what's this

evilsocket commented 1 week ago

because you haven't downloaded the model