search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.66k stars 334 forks source link

koboldcpp.py only sets half of the available threads #72

Closed Myrminki closed 1 year ago

Myrminki commented 1 year ago

I had to manually set the thread count in the default_threads variable. I don't know if it's something that can be set with an argument and I like it because it helps with stability, but I should be able to use all threads if I want to.

LostRuins commented 1 year ago

This is by design, as we found that using more threads can actually lead to a worse performance. There is a --threads parameter you can use when running the program to override this.

Myrminki commented 1 year ago

Well in my case I get a nice performance boost when using all threads, thanks for replying and thanks for the work y'all are doing.