search

marella / chatdocs

Chat with your documents offline using AI.
MIT License
684 stars 99 forks source link

Incomplete response #43

Closed ChrisBNECBT closed 1 year ago

ChrisBNECBT commented 1 year ago

The response of a query is incomplete. ChatGPT3.5 allows users to prompt it to continue with the response in a new prompt and the team at Oobabooga added a "Continue" button to their WebUI to do the same.

There is no way to continue lost response currently through a new prompt and splitting the prompts into smaller chunks resulted in a lost of context when responding.

I'm unsure whether this is a bug or missing feature and am wondering is there a fix to allow a full response.

I'm new to this.

Screenshot 2023-07-09 at 08-47-17 ChatDocs

marella commented 1 year ago

You can increase max_new_tokens (default is 256) to get longer responses:

ctransformers:
  config:
    max_new_tokens: 500 # increase this value

Related to #28