feat(chat command): Word editing

rendezqueue / rendezllama

CLI for llama.cpp with various commands to guide, edit, and regenerate tokens on the fly.

ISC License

10 stars 1 forks source link

Closed grencez closed 1 year ago

grencez commented 1 year ago

[x] Command to backtrack by 1 word.
[x] ~~Command to generate 1 word.~~ Can already do this by setting /sentence_token_limit 1 and pressing enter until a full word is generated.
[x] ~~Command to insert 1 word.~~ Can already do this by setting /sentence_token_limit 1 and typing the word you want to insert with a preceding space.

I guess the commands should support more than just 1 word. Token-level editing should be supported too.

grencez commented 1 year ago

Hrm. I'm not 100% sure how to make it start a new word, only a new token. It might add to the end of a word and be hard to delete!

Probably can be avoided by trial and error. Like, if the generated token doesn't have a space prefix, zero out its probability and try again.

grencez commented 1 year ago

Simplified this feature to just the token & word deletion commands. We can get by without special insert and generate commands.

grencez commented 1 year ago

Ugh, I meant to type "feat(chat command)" on that commit. Oh well, can't fix it now.