nextcloud / assistant

✨ Nextcloud Assistant
GNU Affero General Public License v3.0
35 stars 6 forks source link

prompt_style is exposed in chat with AI #93

Open zydisney opened 5 months ago

zydisney commented 5 months ago

Which version of assistant are you using?

1.1.0

Which version of Nextcloud are you using?

v29.0.2

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

No response

Describe the Bug

prompt_style is exposed in 'chat with AI', 'free prompt' response empty.

Chat with AI


admin Hello

Assistant

im_start|>assistant
Hello! I'm welling to help! Tell me anything you want me to do!

admin

Who are you?

Assistant

Hello! I'm welling to help! Tell me anything you want me to do!
Who are you?
assistant:

admin

What can you do?

Assistant

Hello! I'm welling to help! Tell me anything you want me to do!
Who are you?
Hello! I'm welling to help! Tell me anything you want me to do!
Who are you?
assistant:
What can you do?
assistant:

im_start|>assistant should not be displayed. The sentences are repeated over and over again. The model using a CHATML prompt_style. However, such problem doesn't happen when I use Gradio, Postman, Langchain to test the API.

    "prompt_style": {
        "style_name": "CHATML",
        "system_prompt": "",
        "roles": [
            "<|im_start|>user",
            "<|im_start|>assistant"
        ],
        "intra_message_sep": "<|im_end|>",
        "inter_message_sep": "",
        "stop": [
            "<|endoftext|>",
            "<|im_start|>",
            "<|im_end|>",
            "<|im_sep|>"
        ],
        "stop_token_ids": [
            2,
            6,
            7,
            8
        ]
    }

Expected Behavior

postman API test

# post
{
    "messages": [
        {"role": "system", "content": "\nYou are ChatGPT, a large language model \nKnowledge cutoff: 2023-06-25\nCurrent model: Yi-1.5-chat-0621\nCurrent time: 2023-09-21 10:52:21\n User‘s name is admin"}, 
        {"role": "user", "content": "Hello"}, 
        {"role": "assistant", "content": "Hello! What can I do for you?"}, 
        {"role": "user", "content": "Who are you?"}],
    "stream": false,
    "model": "Yi-1.5-chat-0621",
    "temperature": 0.7,
    "presence_penalty": 0,
    "frequency_penalty": 0,
    "top_p": 0.9
}
# receive
{
    "id": "chat85bb0374-21a6-4094-b1f2-6096c4390445",
    "object": "chat.completion",
    "created": 1719193384,
    "model": "Yi-1.5-chat-0621",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "I'm a large language model."
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 135,
        "completion_tokens": 23,
        "total_tokens": 158
    }
}

The prompt style doesn't appear in normal post response. How to fix the issue.

To Reproduce

.