search

LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.36k stars 312 forks source link

Can't log koboldcpp? #789

Closed Sazu-bit closed 2 months ago

Sazu-bit commented 2 months ago

My issue is that I want to store the output from the terminal as I want to automate some things that is quite frankly a hassle to deal with each time. I'm using a bash script to do part of it.

gnome-terminal -- koboldcpp --config /path/to/config.kcpps &

This works perfectly well, but I want the output in a file so I'm trying:

gnome-terminal -- sh -c "koboldcpp --config /path/to/config.kcpps |& tee /path/to/file.log" &

The latter looks like it works, it's certainly taking up my RAM and VRAM, the loading progresses (inordinately quickly so there's definitely something going wrong) to 

llama_build_graph: non-view tensors processed: 844/844
llama_new_context_with_model: compute buffer total size = 1060.01 MiB

All the numbers look correct. I expect it after this point to tell me about the mounted endpoint and where to connect, but it doesn't it just stops dead right there. No CPU usage, RAM and VRAM is still held. I suspect that there's something in the process that's outputting somewhere its not supposed to because of the pipe, is there anything I can do to resolve this while also keeping the output?

On a side note, I have to forcibly clear VRAM myself if I close the app (I run it on my daily driver) using "sync; echo 2 > /proc/sys/vm/drop_caches", this is admittedly a pita as it requires sudo and I'm surprised that the koboldcpp doesn't do this itself.

(Note: koboldcpp is an alias for '/path/to/kobaldcpp')

Sazu-bit commented 2 months ago

Fixed it, basically a none-issue from the looks of thing. I'm using a preceding line to check that it has loaded (notably the Llama_new_context_with_model), then skipping over the rest of my activities.