search

kwaroran / RisuAI

Make your own story. User-friendly software for LLM roleplaying
https://risuai.net
GNU General Public License v3.0
793 stars 140 forks source link

[Feature Request] Migrate to Cohere v2/chat endpoint #633

Open cloak1505 opened 1 month ago

cloak1505 commented 1 month ago

EDIT: SEE NEXT COMMENT. v2 API is easier to deal with. Frontends still need to manually migrate from v1 to v2. Below is just a weird bug report for v1.

Windows client v134.1.0

Problem: When no context is set (e.g. system prompt, persona, character) and Cohere API is selected, the [Start a new chat] system message gets malformed, and the first message is lost if it belongs to user role.

RisuAI empty context Cohere bug

However, the first message as user returns if you continue the chat so there's 3+ messages. Notably, when there's a preamble and first message as user, [Start a new chat] is assigned to a message with system role instead of preamble.

The bug does not occur if the first message belongs to assistant role, and [Start a new chat] appears in preamble. Edit: First message as assistant also creates a [Start a new chat] system message in chat_history when there is a preamble.

Note that SillyTavern always appends [Start a new Chat] (this line can be edited or disabled) to the end of preamble, which is the expected behavior.

cloak1505 commented 1 month ago

Heads up, they just released a more standardized API today. v1 was so screwy! 🎉 https://docs.cohere.com/reference/chat (edit: new link, old one 404's) I.e. the usual messages parameter and system role, removing the preamble, chat_history, and message junk.

Last message still needs to be user role or else it errors.

cloak1505 commented 1 month ago

Both ST and OpenRouter have migrated to v2. Consecutive same role messages do not need to be squashed (though you can), it just werks. System messages can be anywhere. Simply, the last message is user role.

Most of the time it's safe to simply convert last message to user whether it was assistant or system. Group chat would naturally have an instruction to reply as {{char}}. An assistant message written in third person converted to user is OK in the event the user presses Send instead of Continue, which has continue prompt. OK if there are name prefixes, but Risu doesn't seem to have an Include Names option?

The only time when converting to user may be screwy without attaching "Continue" user placeholder is solo no-name + assistant written in first person as last message.