search

LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with a KoboldAI UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.41k stars 318 forks source link

Severe littering of TEMP folder, hundreds of gygabites over a few days of using kobold #768

Closed brokofankone closed 3 months ago

brokofankone commented 3 months ago

There is some type of issue with the way kobold cpp launches. It creates culbast dlls and other requirements in new folders over and over again, in the temp C folders, which is also not detected by disk cleanup windows utility.

After a week of using it I can easily rack up 100-300GB of temp files.

FgmWHSZ6z_iCURd_IbFOB

You can see the kobold traces, it creates folders named in such a format: -zgfKrxckNUSVgW2MGJCp

Creation of new temp folder on fresh model load: SZisgXjh8ZmcqJ3SeLSXL

This problem was investigated a bit in this HF thread, as I originally assumed it's a quantization script that does it, but we confirmed it is kobold doing it:

https://huggingface.co/FantasiaFoundry/GGUF-Quantization-Script/discussions/8#6612ad190b91dd969677bba1

LostRuins commented 3 months ago

What windows are you running, and how do you normally start & exit koboldcpp?

Spacellary commented 3 months ago

I confirmed this happening here.

Setup: Windows 11 Pro Ryzen 5 1600 AF (6/12) GTX 1070 Ti (8GB)

brokofankone commented 3 months ago

Windows 10 Pro (up-to-date) Closing kobold terminal with X I launch it with the self-contained cuda exe

Spacellary commented 3 months ago

I run KCPP from the Windows Terminal directly.

koboldcpp "model.gguf" --multiuser --gpulayers 99 --contextsize 12288 --port 6969 --blasbatchsize 256 --usecublas --remotetunnel --highpriority --quiet

I interrupt the script with a CTRL + C.

You could say that's not necessarily a graceful exit.

I'm your 16th biggest fan btw, Concedo! Just trust me on that one.

askmyteapot commented 3 months ago

So do we need to change from one file pyinstaller to one folder and use .zip to prevent

LostRuins commented 3 months ago

No, I will come up with a way to clean up the orphaned files.

LostRuins commented 3 months ago

Hello, this has now been fixed in the latest version. Cleanup of old koboldcpp pyinstallers temp dirs should happen on launch.

brokofankone commented 3 months ago

Thank you! 🫡

Spacellary commented 3 months ago

Thank you Concedo-senpai.

henk717 commented 3 months ago

Ill also comment a bit on why it happens to begin with, the pyinstaller does indeed extract tempoary files it needs to temp and it has an instance that monitors for this and cleans it up. This works well and reliable for me on Windows 10 on the regular terminal, but in the modern Windows Terminal microsoft kills the process more aggressively so then the cleanup routine is never executed.

So the fact the files are left over isn't a Koboldcpp bug, but the fact Koboldcpp is being killed so forcefully by the terminal. I think we can close this issue now that the workaround is in place.

brokofankone commented 3 months ago

Thank you for the extra explanation, makes sense! I'm closing it considering the fix is deployed and working.