search

marella / chatdocs

Chat with your documents offline using AI.
MIT License
696 stars 101 forks source link

Any way to set temperature in the yaml file? #40

Closed Ananderz closed 1 year ago

Ananderz commented 1 year ago

Hi,

I have seen quite a bit of hallucinations with the model. I am trying to find a way to set temperature. Is there something I can do in the yml file?

marella commented 1 year ago

Hi, you can set temperature for ctransformers/hf/gptq models using:

ctransformers:
  config:
    temperature: 0.8

huggingface:
  pipeline_kwargs:
    temperature: 0.8

gptq:
  pipeline_kwargs:
    temperature: 0.8