search

rendezqueue / rendezllama

CLI for llama.cpp with various commands to guide, edit, and regenerate tokens on the fly.
ISC License
10 stars 1 forks source link

feat(option): Adding a space before each line #6

Closed grencez closed 1 year ago

grencez commented 1 year ago

I've been adding a space before most prompt lines so that the first word (usually a name) is tokenized the same way as when it appears in a sentence. This makes intuitive sense, but I wonder if the different tokenization helps instill a useful meaning.

In any case, it would be more natural to write prompt lines without the leading space and let the code reformat as desired.

grencez commented 1 year ago

We seem to get better quality by not adding the leading space, so I changed the default. This is consistent with llama.cpp anyway.