search

LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.35k stars 312 forks source link

KoboldCPP Settings not saving? #876

Open ElfrinoGem opened 1 month ago

ElfrinoGem commented 1 month ago

IMPORTANT NOTE: It's not Cuda12 at all, it's something wonky with my Kobold settings, they don't save!

update:

Hey, I found something super strange. I don't think it's Cuda at all but something weird with my KoboldCPP ( I don't know if this is effecting others or just me though).

Each time I load Kobold I need to manually choose a preset and then configure the settings to how I want them. Kobold won't recognize my saved settings even though they 'seem' to be set correctly when i load in. Basically for kobold to see my settings I have to manually adjust them each time I enter the program.

LostRuins commented 1 month ago

I doubt there should be much difference. Are you sure your generation settings are the same? Try using MMQ, and disable Flash Attention on both. You can also try using a fixed seed.

ElfrinoGem commented 1 month ago

I had been using MMQ but I'll try disabling Flash Attention and trying a fixed seed and see how it goes. Thankyou for the tips.

LostRuins commented 1 month ago

Settings are saved in the JSON file (or autosave, if you dont load from file). are you perhaps overwriting them with a different JSON?

ElfrinoGem commented 1 month ago

Ahh, I have a whole bunch of Kobold versions in the same directory and I use them interchangeably. Do you think they could be interfering with each other? Where is the JSON file stored?

LostRuins commented 1 month ago

Are you using the built-in save slots?

ElfrinoGem commented 1 month ago

Yes I use them constantly to save my stories. Ahh, got it, settings are saved with the story. Thankyou. Is it possible to save your own settings preset? (aka Mayday, Goodwinds, etc) so it appears in the drop down window?

LostRuins commented 1 month ago

Nope, the presets are fixed. However, you can easily save your desired generation settings along with each story.

You can also selectively choose what to load by enabling "Advanced Load" in settings.

ElfrinoGem commented 1 month ago

Thankyou.