Error Loading LLAMA 3.1 llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 292, got 291[BUG]:

SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

https://scisharp.github.io/LLamaSharp

MIT License

2.48k stars 331 forks source link

Error Loading LLAMA 3.1 llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 292, got 291[BUG]: #875

Open emulk opened 1 month ago

emulk commented 1 month ago

Description

[BUG]: Error Loading LLAMA 3.1 llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 292, got 291

Reproduction Steps

Error Loading LLAMA 3.1 .gguf model

Environment & Configuration

Operating system: Windows 10
.NET runtime version: .net 8
LLamaSharp version:
CUDA version (if you are using cuda backend):
CPU & GPU device: 13th Gen Intel(R) Core(TM) i7-13850HX, 2304 Mhz, 20 Core(s), 28 Logical Processor(s)

Known Workarounds

No response

martindevans commented 1 month ago

Could you try it with this branch, that should include the llama.cpp fixes for llama 3.1.

CastIehard commented 1 month ago

I am having the exact same problem also using Windows 10.:

llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 292, got 291
llama_load_model_from_file: failed to load model

martindevans commented 1 month ago

Was that with the PR I linked above? It's not expected that llama 3.1 will work with master or the current nuget packages.

m0nsky commented 1 month ago

@emulk @CastIehard this should now be fixed in the LLamaSharp 0.15 release from 3 days ago.