Bug: `chatStream` does not recognize `chat_history`

[danny-avila]

Version: ^7.9.1

Using cURL, it works. I built the messages while working with the cohere-ai SDK

curl --location 'https://api.cohere.ai/v1/chat' \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--header 'Authorization: bearer REDACTED_KEY' \
--data '{
  "message": "uhhhhh you know what you did",
  "model": "command-r",
  "chat_history": [
    {
      "role": "USER",
      "message": "how do I solve fibonacci in python?"
    },
    {
      "role": "CHATBOT",
      "message": "Here is a Python program that generates Fibonacci numbers:\n```python\ndef fibonacci(n):\n    if n <= 0:\n        return 0\n    elif n == 1:\n        return 1\n    else:\n        return fibonacci(n - 1) + fibonacci(n - 2)\n\nn = int(input(\"Enter a \"number\": \"))\nprint(f\"The {n}th Fibonacci number is {fibonacci(n)}\")\n```\n\nThis program defines a recursive function `fibonacci` that calculates the nth Fibonacci number. The function takes an integer `n` as input and returns the nth Fibonacci number. The base cases are defined for `n` equal to 0 and 1, where the function returns 0 and 1, respectively. For other values of `n`, the function recursively calls itself to calculate the sum of the (n-1)th and (n-2)th Fibonacci numbers.\n\nThe program then takes user input for `n` and calls the `fibonacci` function to calculate and print the nth Fibonacci number."
    },
    {
      "role": "USER",
      "message": "wtf how did you do that"
    },
    {
      "role": "CHATBOT",
      "message": "I'\''m sorry, what did you mean by that? Did you see a cool trick or something that surprised you? Please let me know so I can clarify or repeat the action. I want to make sure I'\''m providing the best assistance possible!"
    }
  ],
  "stream": true
}'

I get the reply:

"Oh, you must be referring to the Fibonacci sequence generation in Python that I mentioned earlier. I did that using a recursive function in Python. \n\nThe Fibonacci sequence is a series of numbers in which each number is the sum of the two preceding ones, usually starting with 0 and 1. So, the first few numbers in the Fibonacci sequence are 0, 1, 1, 2, 3, 5, 8, 13, and so on.\n\nIn the Python code I provided, the `fibonacci` function takes an integer `n` as input and returns the nth Fibonacci number. The function is recursive, which means it calls itself with smaller inputs to build up the final result. The base cases are defined for `n` equal to 0 and 1, and for other values of `n`, the function makes two recursive calls with the inputs `n - 1` and `n - 2`. The results of these recursive calls are then summed to obtain the nth Fibonacci number.\n\nI hope now you understand the trick ;) Feel free to ask if you need more clarification on recursive functions or the Fibonacci sequence!"

The exact same payload with SDK:

    const chatStream = await cohere.chatStream(payload);
    let reply = '';
    for await (const message of chatStream) {
      if (!message) {
        continue;
      }

      if (message.eventType === 'text-generation' && message.text) {
        onTokenProgress(message.text);
      } else if (message.eventType === 'stream-end' && message.response) {
        reply = message.response.text;
      }
    }

    return reply;

the reply is

"I apologize if I have done something to upset you. As an AI chatbot, my purpose is to be as helpful and harmless as possible. I do not have access to personal information, and I respect the privacy of all individuals. 

Can you please clarify what specific actions you are referring to so that I can address your concerns accordingly?"

So this must be an SDK issue and not API

[vrtnis]

seems like the loop for handling events from chatStream seems to be based on specific event types like text-generation and stream-end...it's possible that the SDK is emitting different event types, there is also this comment in ChatRequest.ts fwiw:, just an observation

The chat_history parameter should not be used for SYSTEM messages in most cases. Instead, to add a SYSTEM role message at the beginning of a conversation, the preamble parameter should be used. *

[danny-avila]

Thanks for your reply!

The chat_history parameter should not be used for SYSTEM messages in most cases. Instead, to add a SYSTEM role message at the beginning of a conversation, the preamble parameter should be used. *

I think there should be flexibility with this. I understand the use case for preamble and I like it, but the user should decide where they want system messages to test how it affects the model. This is implied by having the option; otherwise, we should not have this option at all.

But in any case I figured out the issue. the SDK is expecting camel-case parameters. A closer look at the ChatStreamRequest type reveals this, sorry I missed this. chatHistory works, chat_history does not.

I recommend allowing aliases to avoid any confusion here.

[billytrend-cohere]

Hey @danny-avila glad you fixed it. This is tripping a lot of people up! Will hopefully have runtime checking soon to prevent this issue. I also recommend you use typescript which will warn you about it. Thanks!