search

lmstudio-ai / configs

LM Studio JSON configuration file format and a collection of example config files.
MIT License
188 stars 32 forks source link

Impossible increase CPU threads! #19

Open Groboto opened 7 months ago

Groboto commented 7 months ago

I change oin the json file the thread to 8, but after finish process my prompt, inform 4threads. The CPU usage of my 3950x on the LLM studio CPU metter never go over 49%

Why cant push more the CPU in order to speed up the process?

im7860 commented 7 months ago

You know what, I'm facing the same problem. I'm only using half of my computer's processing power and there isn't an option to increase the CPU threads. (Looks like it was removed -- because the old version had it) Synlink 2

Groboto commented 7 months ago

Well is possible change that directly form the json configuration, finally I already test and not make any difference or speed up anything. Try increase up to 8, maybe you have a very little improvement, but is really near to null.

No waste your time like me lol, trying to speedup only with CPU. pickup dual or cuad 3090 or 4090, or better, that is the way, and be careful about that, you need every card plugged as minimum at PCIe 3.0 at 8 lanes, or 4lanes under PCIe 4.0, any PCIe speed under that that values create a big bottleneck. That amount of lanes on PCIe is only available in few pro boards or you need to go with epic CPU Threadripper etc. SO be careful if you plan also builds something with multiple GPUs. The software really use all my CPU cores, but for some reason just push up to 50% only. I assume there is a point where parallelization starts to add penalty instead benefit, that happens with CPU and also with GPU.