search

nomic-ai / gpt4all

GPT4All: Chat with Local LLMs on Any Device
https://gpt4all.io
MIT License
66.09k stars 7.28k forks source link

[Feature] Custom stopping word list #2430

Open ElvishScout opened 2 weeks ago

ElvishScout commented 2 weeks ago

Feature Request

On some llama models text generation won't stop after hitting the <|im_end|> tag. There is a workaround by adding <|im_end|> to the stopping word list on other platforms such as llama-cpp-python.

I wonder how this problem can be fixed on GPT4All.

image

ThiloteE commented 2 weeks ago

Related stop token issue, which stop word list will be a workaround for: https://github.com/nomic-ai/gpt4all/issues/2167#issuecomment-2116312336

Competes with https://github.com/nomic-ai/gpt4all/issues/2439