Open sweihub opened 1 year ago
Hi,
Thanks for the simple guide, however, it's not working, anything mismatched?
./main -m models/ggml-vicuna-13B-1.1-q5_1.bin --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-vicuna-v1.txt
output
main: build = 732 (afd983c) main: seed = 1696926741 llama.cpp: loading model from models/ggml-vicuna-13B-1.1-q5_1.bin llama_model_load_internal: format = ggjt v3 (latest) llama_model_load_internal: n_vocab = 32000 llama_model_load_internal: n_ctx = 512 llama_model_load_internal: n_embd = 5120 llama_model_load_internal: n_mult = 256 llama_model_load_internal: n_head = 40 llama_model_load_internal: n_layer = 40 llama_model_load_internal: n_rot = 128 llama_model_load_internal: ftype = 9 (mostly Q5_1) llama_model_load_internal: n_ff = 13824 llama_model_load_internal: n_parts = 1 llama_model_load_internal: model size = 13B llama_model_load_internal: ggml ctx size = 0.00 MB error loading model: llama.cpp: tensor 'norm.weight' is missing from model llama_init_from_file: failed to load model llama_init_from_gpt_params: error: failed to load model 'models/ggml-vicuna-13B-1.1-q5_1.bin' main: error: unable to load model
Really sorry, haven't had the time to keep this guide up to date. Just committed, let me know if it works
Hi,
Thanks for the simple guide, however, it's not working, anything mismatched?
output