search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.35k stars 364 forks source link

Crash while loading Minitron-4B-Width-Base #1116

Closed Himickoff closed 2 months ago

Himickoff commented 2 months ago

Koboldcpp 1.74 Default settings Windows 10 Gtx 1650 super

Logs: llama_model_load: error loading model: check_tensor_dims: tensor 'rope_freqs.weight' has wrong shape; expected 64, got 48,'▼?▬zgllama_load_model_from_file: failed to load model Traceback (most recent call last): File "koboldcpp.py", line 4451, in main(parser.parse_args(),start_server=True) File "koboldcpp.py", line 4099, in main loadok = load_model(modelname) File "koboldcpp.py", line 871, in load_model ret = handle.load_model(inputs) OSError: exception: access violation reading 0x0000000000001894 [7224] Failed to execute script 'koboldcpp' due to unhandled exception!

LostRuins commented 2 months ago

You are probably using a bad quant. There has been many bad minitron 4b width base quants.

Please try this one https://huggingface.co/mradermacher/Minitron-4B-Base-GGUF/tree/main

Himickoff commented 2 months ago

Thanks, it's all working now!