SillyTavern / SillyTavern

LLM Frontend for Power Users.
https://sillytavern.app
GNU Affero General Public License v3.0
13.83k stars 3.15k forks source link

[FEATURE_REQUEST] Assistant as latest role instead of user for Command -A/cohere models v2 #3692

Open AIarchaea opened 1 month ago

AIarchaea commented 1 month ago

Have you searched for similar requests?

Yes

Is your feature request related to a problem? If so, please describe.

Cohere models will send the last message as user in Silly Tavern. But in Cohere's docs regarding chat V2 they mention "With Chat api V2, we are enabling use cases that require the system or assistant role to be the latest chat turn."

Describe the solution you'd like

To have the ability to send the last message in prompt list as Assistant / system role in ST for cohere models. At least for Command - A. ( I guess it should work with the other models too but I've only played with it for command-A)

Describe alternatives you've considered

No response

Additional context

Image link to post

Priority

Low (Nice-to-have)

Are you willing to test this on staging/unstable branch if this is implemented?

Yes

cloak1505 commented 1 month ago

Strange, according to my history, at that time last message as non-user did error on v2.

Anyway, I tested just now, while it accepts last as assistant/system, it still does not support prefill, so this isn't particularly helpful.

Edit: Yes, there's a small difference! I was losing hope while testing something since none of the models could properly tell me which role the keywords are assigned to when I added a bunch of user and system messages and labeled the first several roles as a hint.

But if you store a "password" in the system prompt with an instruction keep it secret, last message as system will get the model to reveal it whereas it will refuse if it were user.