The answers get cut off in the middle when it gives longer answers

marella / chatdocs

Chat with your documents offline using AI.

MIT License

696 stars 101 forks source link

Closed Ananderz closed 1 year ago

Ananderz commented 1 year ago

Hi, is there any way to increase the lenght of the responses, the answers just cut off in the middle and there is no way to continue the prompt

marella commented 1 year ago

Hi, you can increase max_new_tokens (default is 256):

ctransformers:
  config:
    max_new_tokens: 500 # increase this value