The LLM doesn't use all the cores available or the number of cores is not customizable (defaults to 4 when using privateGPT.py)

FCL-LM / privateGPT

Interact privately with your documents using the power of GPT, 100% privately, no data leaks

Apache License 2.0

0 stars 0 forks source link

Closed josura closed 1 year ago

josura commented 1 year ago

Is your feature request related to a problem? Please describe. The models do not use all the CPU available.

Describe the solution you'd like Add the n_threads parameter to the model call and take a parameter as an argument or an environment variable

Describe alternatives you've considered Use all cores

Additional context The model is very slow

josura commented 1 year ago

Closing the issue, resolved in pull request #5