LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.66k stars 334 forks source link

[User] Error loading model: unrecognized tensor type 4 #106

Closed mainsplainer closed 1 year ago

mainsplainer commented 1 year ago

Failed to load q4_2 model from here -> https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/tree/main

D:\Projects\REPOS\CPU Text Generation>koboldcpp --useclblast 0 0 --smartcontext --stream
Welcome to KoboldCpp - Version 1.10
Attempting to use CLBlast library for faster prompt ingestion. A compatible clblast.dll will be required.
Initializing dynamic library: koboldcpp_clblast.dll
For command line arguments, please refer to --help
Otherwise, please manually select ggml file:
Loading model: D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin
[Parts: 1, Threads: 5, SmartContext: True]

---
Identified as LLAMA model: (ver 3)
Attempting to Load...
---
System Info: AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |
llama.cpp: loading model from D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin
error loading model: unrecognized tensor type 4

llama_init_from_file: failed to load model
llama_load_model: error: failed to load model 'D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin'
Load Model OK: False
Could not load model: D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin
LostRuins commented 1 year ago

Please update to the latest version and try again

mainsplainer commented 1 year ago

Thanks! Updating to the latest version worked and the issue has been resolved.