Closed mainsplainer closed 1 year ago
Failed to load q4_2 model from here -> https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/tree/main
D:\Projects\REPOS\CPU Text Generation>koboldcpp --useclblast 0 0 --smartcontext --stream Welcome to KoboldCpp - Version 1.10 Attempting to use CLBlast library for faster prompt ingestion. A compatible clblast.dll will be required. Initializing dynamic library: koboldcpp_clblast.dll For command line arguments, please refer to --help Otherwise, please manually select ggml file: Loading model: D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin [Parts: 1, Threads: 5, SmartContext: True] --- Identified as LLAMA model: (ver 3) Attempting to Load... --- System Info: AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 | llama.cpp: loading model from D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin error loading model: unrecognized tensor type 4 llama_init_from_file: failed to load model llama_load_model: error: failed to load model 'D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin' Load Model OK: False Could not load model: D:\Projects\REPOS\CPU Text Generation\models\ggml-vicuna-13b-1.1-q4_2.bin
Please update to the latest version and try again
Thanks! Updating to the latest version worked and the issue has been resolved.
Failed to load q4_2 model from here -> https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/tree/main