search

cocktailpeanut / dalai

The simplest way to run LLaMA on your local machine
https://cocktailpeanut.github.io/dalai
13.09k stars 1.42k forks source link

How to utalize more threads? #403

Open mehmetpakkabaggen opened 1 year ago

mehmetpakkabaggen commented 1 year ago

In the output after you exit the application, it says: '--seed 355555556 --threads 4 --n_predict 200 --model models/7B/ggml-model-q4_0.bin --top_k 40.' As you can see, it only uses 4 threads, and I have 12. How do I make it utilize all of them?

supportend commented 1 year ago

See in the upper right of the web gui, there you should be able to set the threads,