LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.36k stars 312 forks source link

Cannot load a saved config in 1.63 sometimes #806

Closed aleksusklim closed 2 months ago

aleksusklim commented 2 months ago

The KCPPS config:

{"model": null, "model_param": "Lexi-Llama-3-8B-Uncensored-Q5_K_M.gguf", "port": 5001, "port_param": 5001, "host": "", "launch": true, "config": null, "threads": 8, "usecublas": ["normal", "0", "mmq"], "usevulkan": null, "useclblast": null, "noblas": false, "gpulayers": 0, "tensor_split": null, "contextsize": 4096, "ropeconfig": [0.0, 10000.0], "blasbatchsize": 512, "blasthreads": null, "lora": null, "smartcontext": false, "noshift": true, "bantokens": null, "forceversion": 0, "nommap": false, "usemlock": true, "noavx2": false, "debugmode": 0, "skiplauncher": false, "hordeconfig": null, "onready": "", "benchmark": null, "multiuser": 1, "remotetunnel": false, "highpriority": false, "foreground": false, "preloadstory": null, "quiet": false, "ssl": null, "nocertify": false, "sdconfig": null, "mmproj": null, "password": null, "ignoremissing": false, "chatcompletionsadapter": null}

I open this config: image

I change Use CuBLAS to Use OpenBLAS: image

I save the config:

{"model": null, "model_param": "Lexi-Llama-3-8B-Uncensored-Q5_K_M.gguf", "port": 5001, "port_param": 5001, "host": "", "launch": true, "config": null, "threads": 8, "usecublas": null, "usevulkan": null, "useclblast": null, "noblas": false, "gpulayers": 0, "tensor_split": null, "contextsize": 4096, "ropeconfig": [0.0, 10000.0], "blasbatchsize": 512, "blasthreads": null, "lora": null, "smartcontext": false, "noshift": true, "bantokens": null, "forceversion": 0, "nommap": false, "usemlock": true, "noavx2": false, "debugmode": 0, "skiplauncher": false, "hordeconfig": null, "onready": "", "benchmark": null, "multiuser": 1, "remotetunnel": false, "highpriority": false, "foreground": false, "preloadstory": null, "quiet": false, "ssl": null, "nocertify": false, "sdconfig": null, "mmproj": null, "password": null, "ignoremissing": false, "chatcompletionsadapter": null}

I restart and load the config: image

***
Welcome to KoboldCpp - Version 1.63
For command line arguments, please refer to --help
***
Exception in Tkinter callback
Traceback (most recent call last):
  File "tkinter\__init__.py", line 1892, in __call__
  File "customtkinter\windows\widgets\ctk_button.py", line 554, in _clicked
    self._command()
  File "koboldcpp.py", line 2348, in load_config
  File "koboldcpp.py", line 2234, in import_vars
TypeError: argument of type 'NoneType' is not iterable

The error prevents loading of all fields, and not just the one it considers broken!

Can you:

  1. Fix whatever caused the error here; and
  2. Make loading more tolerant to incomplete/bad configs? (As long as it is valid JSON)
LostRuins commented 2 months ago

Alright found the issue, I'll fix it. Generally the kcpps file should be fault tolerant, this is one edge case that was missed.

LostRuins commented 2 months ago

Hi, Should be fixed in the latest version

aleksusklim commented 2 months ago

Yup, it's working!