Bug description:
I've noticed multiple cases of the user's name changing to "You" and the bot's name to "Bot" unexpectedly, ruining generations. This seems to happen whenever an auto-generation is invoked (instead of a normal generation started by sending a user message normally), be it through a command, summarization, or group chat's self responses.
Steps to reproduce the behavior:
Chat normally and check the logs, {{user}} and {{char}} get replaced with user's and character's name
Start an auto-generation, e. g. by using the command /continue, /bgauto, or any other way to make the AI respond on its own
Check the logs, {{user}} is now "You" and {{char}} is "Bot", despite the character and user persona having other names
Expected behavior:
{{user}} and {{char}} are always the proper user and character names and don't unintentionally revert to "You" and "Bot".
Logs:
Proxy:
…
{ role: 'system', content: 'Char|User' },
{
role: 'system',
content: '[Continue the following message. Do not include ANY parts of the original message. Use capitalization and punctuation as if your reply is a part of the original message:\n' +
…
Backend (koboldcpp):
Input: {"n": 1, "rep_pen": 1.18, "temperature": 0, "top_p": 0, "top_k": 1, "top_a": 0, "typical": 1, "tfs": 1, "rep_pen_range": 2048, "rep_pen_slope": 0, "sampler_order": [0, 1, 2, 3, 4, 5, 6], "quiet": false, "max_context_length": 2048, "max_length": 250, "prompt": "## Bot\n- You're "Bot" in this never-ending roleplay with "You".\n…\n[Continue the following message. Do not include ANY parts of the original message. Use capitalization and punctuation as if your reply is a part of the original message:\n…", "stop_sequence": ["", "\n##", "\nYou:", "\n# ", "##", "\nInstruction:", "\nUser:"]}
Additional context:
The logs show two system messages, the first being the jailbreak with the user and bot names. The second is the auto-generation prompt, in this example the /continue command's prompt.
Problem: Looks like the proxy gets confused by the additional system message and stops extracting the proper names from the jailbreak prompt, thus reverting to "Bot" and "You" as bot and user names. This ruins generation because the LLM gets confused.
Solution: The proxy should gracefully handle additional system messages and extract the proper names from the jailbreak system message.
Bug description: I've noticed multiple cases of the user's name changing to "You" and the bot's name to "Bot" unexpectedly, ruining generations. This seems to happen whenever an auto-generation is invoked (instead of a normal generation started by sending a user message normally), be it through a command, summarization, or group chat's self responses.
Steps to reproduce the behavior:
Expected behavior: {{user}} and {{char}} are always the proper user and character names and don't unintentionally revert to "You" and "Bot".
Logs: Proxy: … { role: 'system', content: 'Char|User' }, { role: 'system', content: '[Continue the following message. Do not include ANY parts of the original message. Use capitalization and punctuation as if your reply is a part of the original message:\n' + …
Backend (koboldcpp): Input: {"n": 1, "rep_pen": 1.18, "temperature": 0, "top_p": 0, "top_k": 1, "top_a": 0, "typical": 1, "tfs": 1, "rep_pen_range": 2048, "rep_pen_slope": 0, "sampler_order": [0, 1, 2, 3, 4, 5, 6], "quiet": false, "max_context_length": 2048, "max_length": 250, "prompt": "## Bot\n- You're "Bot" in this never-ending roleplay with "You".\n…\n[Continue the following message. Do not include ANY parts of the original message. Use capitalization and punctuation as if your reply is a part of the original message:\n…", "stop_sequence": ["", "\n##", "\nYou:", "\n# ", "##", "\nInstruction:", "\nUser:"]}
Additional context: The logs show two system messages, the first being the jailbreak with the user and bot names. The second is the auto-generation prompt, in this example the /continue command's prompt.
Problem: Looks like the proxy gets confused by the additional system message and stops extracting the proper names from the jailbreak prompt, thus reverting to "Bot" and "You" as bot and user names. This ruins generation because the LLM gets confused.
Solution: The proxy should gracefully handle additional system messages and extract the proper names from the jailbreak system message.