A setting to allow more cpu threads, if possible.

[kyle215ps3]

Is their a way for it to use more threads? I see it uses 8 but not more. I have 16 in total.

[ItsPi3141]

Performance will be worse if you use all 16 cores. Your OS will start to kinda die and it will make everything lag.

[jrhartono]

I am using 5900x, there is 8 more thread available, is it possible to allow more core? I want to use 22 out of the 24 threads available. Or is it need to be the power of two? like 8 16 32 threads?

[ItsPi3141]

I am using 5900x, there is 8 more thread available, is it possible to allow more core? I want to use 22 out of the 24 threads available. Or is it need to be the power of two? like 8 16 32 threads?

It works better with it's a power of 2. Back when llama.cpp just first came out, we tested it and found out that on a 32-core CPU, the performance, ranked from best to worst was the following:

• 16 cores
• 32 cores
• 24 cores

So it's better if it's a power of 2. And if it isn't a server (most people aren't running electron apps on servers), it will be slower if it uses all of the threads.

[agreenbhm]

@ItsPi3141 - I've got a server with 60 cores and it shows as using 32/60 threads. Based on that performance test you listed, is 16 better than 32? My virtual host has 72 cores so technically I could allocate 64 threads to the app and still have plenty to spare for the OS; would that be better than 16 or 32? Or is going past 16 going to make things worse?

[ItsPi3141]

@ItsPi3141 - I've got a server with 60 cores and it shows as using 32/60 threads. Based on that performance test you listed, is 16 better than 32? My virtual host has 72 cores so technically I could allocate 64 threads to the app and still have plenty to spare for the OS; would that be better than 16 or 32? Or is going past 16 going to make things worse?

If your server/computer has 60 cores, it should use 32. If it has 72, then it should use 64.

[agreenbhm]

Thanks!