search

KoboldAI / KoboldAI-Client

https://koboldai.com
GNU Affero General Public License v3.0
3.45k stars 743 forks source link

Kobold AI not utilize all cpus #408

Closed romi1996 closed 9 months ago

romi1996 commented 9 months ago

I have a system that has two running CPUs at the same time (36 cores, 72 threads) (2 NUMA Nodes) Kobold AI mode: CPU Mode only When running Kobold AI, it seems to just select the second node and run with it, while the first Node is left idle. Is there anyway to run it on a system that contains more than one CPU? Thank you.

RecoveredApparatus commented 9 months ago

if you are planning to run on bare CPU/CPU only maybe koboldcpp might be a better pick than koboldai as it is meant for CPU usage and also supports formats like gguf.