search

rendezqueue / rendezllama

CLI for llama.cpp with various commands to guide, edit, and regenerate tokens on the fly.
ISC License
10 stars 1 forks source link

feat(option): to customize end-of-sentence tokens #1

Closed grencez closed 1 year ago

grencez commented 1 year ago

Note: "end-of-sentence" here means real sentences, I should have avoided this terminology in the bug to avoid confusing them with the EOS token.

Sentences and chat messages are delimited by specific tokens that you might call "antiprompts" or "reverse prompts". The two concepts (of sentences and chat messages) are pretty fundamental to this project, so we should focus on these classes of delimiters.

End-of-sentence tokens could easily be specified in setting.sxproto. Let's do that.

End-of-message is assumed to be a newline or the EOS token. It could be customized a little bit, but let's handle that in Issue #4.

grencez commented 1 year ago

Original description: A newline always has to be part of the antiprompt, but others should be fine to change. This is a "wontfix" until someone has a need for it. (If so, please comment.)