Context preamble doesn't work

[Adjenz]

Dear xitiomet,

First of all thank you for this nicely developped bot.

I have an issue concerning th e context preamble. It doesn't seem to work as the bot repeats with basics GPT sentences.

I took your exemple to speak in 1337 language only but it works about 1/5 times.

Do you have any idea of what could be wrong?

Thanks

[Adjenz]

Now it repeats what I ask him : 10:31:48 Nejda: wesh 10:31:49 Adjen says "Nejda: wesh" 10:32:03 Nejda: tu fais quoi ? 10:32:04 Adjen says "Nejda: tu fais quoi ?" 10:32:08 bon... 10:32:08 Adjen says "bon..."

[xitiomet]

When you fill in the "systemPreamble" irc-gpt-bot passes that text to OpenAI's API as the "system" message on every API request. How chatGPT responds depends on the model you are using and the content of the system message. From what i understand some models pay more attention or less attention to the system instructions. I did most of my testing on GPT-3.5

See here: https://platform.openai.com/docs/guides/gpt-best-practices/tactic-ask-the-model-to-adopt-a-persona https://platform.openai.com/docs/guides/gpt-best-practices/strategy-test-changes-systematically

If you have logging enabled, take a look at "openai.log" file in the "irc-gpt-bot-logs" (default) directory. It will show the raw content of what's being sent to chatGPT you should be able to determine if your instructions are being sent to chatGPT correctly.

[Adjenz]

Thanks for your reply and the links you provided.

I think I know where the issue comes from. As you can see below the bot send multiples sentences to the API :  

[12:03:59 2023-08-18] [0;92mSending Payload to chatGPT..[0m

[12:03:59 2023-08-18] {   "messages": [       {           "role": "system",           "content": "Reponds uniquement au texte delimite par des antislashs de facon courte sans mettre de majusclues ni de points."       },       {           "role": "user",           "content": "ABCDE says \"soit un petit verre de vin rouge\""       },       {           "role": "user",           "content": "Adjen says \"ok\""       },       {           "role": "user",           "content": "Adjen says \"donc tu bois un demi en deux fois\""       },       {           "role": "user",           "content": "ABCDE says \"voilà\""       },       {           "role": "user",           "content": "Adjen says \"Nejda combien de cl pour un verre de vin ?\""       }   ],   "model": "gpt-3.5-turbo",   "user": "Adjen"

}   This is the reply of the the bot :   [12:04:00 2023-08-18] [0;93mchatGPT RESPONSE:[0m

[12:04:00 2023-08-18] {   "created": 1692353040,   "usage": {       "completion_tokens": 12,       "prompt_tokens": 118,       "total_tokens": 130   },   "model": "gpt-3.5-turbo-0613",   "id": "chatcmpl-7oqciIZ7bqfa5Q9VViw5Wbca61U1c",   "choices": [{       "finish_reason": "stop",       "index": 0,       "message": {           "role": "assistant",           "content": "ABCDE says \"environ 12 centilitres\""       }   }],   "object": "chat.completion"

} Any idea why ? Le 2023-08-17 22:19, Brian @.***> a écrit :

When you fill in the "systemPreamble" irc-gpt-bot passes that text to OpenAI's API as the "system" message on every API request. How chatGPT responds depends on the model you are using and the content of the system message. From what i understand some models pay more attention or less attention to the system instructions. I did most of my testing on GPT-3.5

See here: https://platform.openai.com/docs/guides/gpt-best-practices/tactic-ask-the-model-to-adopt-a-persona https://platform.openai.com/docs/guides/gpt-best-practices/strategy-test-changes-systematically

If you have logging enabled, take a look at "openai.log" file in the "irc-gpt-bot-logs" (default) directory. It will show the raw content of what's being sent to chatGPT you should be able to determine if your instructions are being sent to chatGPT correctly.

—

Reply to this email directly, view it on GitHub (https://github.com/xitiomet/irc-gpt-bot/issues/3#issuecomment-1682919867), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AQJIR7LNFI324JNOZ7GGUODXVZ4GTANCNFSM6AAAAAA3TWOPCE).

You are receiving this because you authored the thread.Message ID: @.***>

[xitiomet]

Interesting, my logs look very similar, except i'm not seeing any "role": "assistant" messages in your log

Example from my logs:

    {
      "role": "user",
      "content": "Adjen says \"hey charlieGPT what do you like to do ?\""
    },
    {
      "role": "assistant",
      "content": "Heh, 'CharlieGPT'! I like it. Alright Adjen, so here's my usual day: First off, I wake up and crack into a fresh can of cat food, that gets the cats rawring and me rowdy. Then, I maybe, I dunno, glue some stuff cause it's good to get a huff of fresh glue in the morning. Gets the ol' engine running, you know. Then I bash some rats, maybe write a new song for the Paddy's Pub (gotta keep that creative juice flowing), spend a few hours diggin' through the trash for treasure, and then it's time for a rigorous game of Nightcrawlers. Oh, not to forget, I take out some time for Bird Law studies as well. And sometimes, I just hang out with Frank doing whatever madness pops into our heads. Can't get enough of that man. So yeah, that's me! Stick around, buddy, you'll fit in real nice!"
    }

I noticed the system message in your log has the instruction "Respond only to text delimited by backslashes" which might be confusing chatGPT's api since no backslashes are actually being sent conversationally. (The backslashes present in the log merely escape the quotations passed as part of the text as part of JSON formatting)

Couple Questions so i can look into it further:

• What name does your bot go by in your setup? (i'm guessing Nejda)
• When multiple people are in a channel the bot wont respond unless directly addressed by its irc nickname, or if it asked a question. maybe try a systemPreamble that has something like, "respond as Nejda"
• Also can you tell me what happens when you try to talk to the bot via a private message?

[Adjenz]

What name does your bot go by in your setup? (i'm guessing Nejda) : Nejda, you're right. When multiple people are in a channel the bot wont respond unless directly addressed by its irc nickname, or if it asked a question. maybe try a systemPreamble that has something like, "respond as Nejda" : Now the bot respond only when we mention it as I turned the depth context to 1 but when it's > 1 it takes random nicknames and quote then. For example if someone says "hi everyone" the bot says "RandomNickname said hi". Also can you tell me what happens when you try to talk to the bot via a private message? I disabled this option.       Le 2023-08-18 14:22, Brian @.***> a écrit :

Interesting, my logs look very similar, except i'm not seeing any "role": "assistant" messages in your log

Example from my logs: { "role": "user", "content": "Adjen says \"hey charlieGPT what do you like to do ?\"" }, { "role": "assistant", "content": "Heh, 'CharlieGPT'! I like it. Alright Adjen, so here's my usual day: First off, I wake up and crack into a fresh can of cat food, that gets the cats rawring and me rowdy. Then, I maybe, I dunno, glue some stuff cause it's good to get a huff of fresh glue in the morning. Gets the ol' engine running, you know. Then I bash some rats, maybe write a new song for the Paddy's Pub (gotta keep that creative juice flowing), spend a few hours diggin' through the trash for treasure, and then it's time for a rigorous game of Nightcrawlers. Oh, not to forget, I take out some time for Bird Law studies as well. And sometimes, I just hang out with Frank doing whatever madness pops into our heads. Can't get enough of that man. So yeah, that's me! Stick around, buddy, you'll fit in real nice!" }

I noticed the system message in your log has the instruction "Respond only to text delimited by backslashes" which might be confusing chatGPT's api since no backslashes are actually being sent conversationally. (The backslashes present in the log merely escape the quotations passed as part of the text as part of JSON formatting)

Couple Questions so i can look into it further:

What name does your bot go by in your setup? (i'm guessing Nejda) When multiple people are in a channel the bot wont respond unless directly addressed by its irc nickname, or if it asked a question. maybe try a systemPreamble that has something like, "respond as Nejda" Also can you tell me what happens when you try to talk to the bot via a private message?

—

Reply to this email directly, view it on GitHub (https://github.com/xitiomet/irc-gpt-bot/issues/3#issuecomment-1683839458), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AQJIR7LOHNZEVSB5SQYRZTLXV5M6PANCNFSM6AAAAAA3TWOPCE).

You are receiving this because you authored the thread.Message ID: @.***>

[Adjenz]

Thanks for your help, it works now :)