streaming chat completions with functions

[waptik]

I noticed that the type defintion for createChatCompletionStream options doesn't accept function_call and functions unlike createChatCompletion that accepts them as options

[lino-levan]

Hm, you're right. Would you be willing to open a PR?

[waptik]

I’d love to but Unfortunately I don’t know how go about making the streaming wok properly

On 7 Oct 2023, at 19:56, Lino Le Van @.***> wrote:

 Hm, you're right. Would you be willing to open a PR?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[lino-levan]

Should just be a type definition change. The streaming should work the same.

[alexewerlof]

Current

export interface ChatCompletionStream {
  id: string;
  object: "chat.completion.chunk";
  created: number;
  choices: {
    index: number;
    delta: {
      name?: string;
      role?: "system" | "assistant" | "user";
      content?: string;
    };
    finish_reason: string;
  }[];
}

Suggestion

export interface ChatCompletionStream {
  id: string;
  object: "chat.completion.chunk";
  created: number;
  choices: {
    index: number;
    delta: {
      name?: string;
      role?: "system" | "assistant" | "user";
      content?: string | null; // sometimes null
      function_call?: {
        name?: string;
        arguments: string;
      };
    };
    finish_reason: string | null; // mostly null
  }[];
}

But I'd actually prefer if we could use FunctionAwareAssistantCompletionMessage or ChatCompletionMessage directly.

Some data from OpenAI GPT 3.5 in case you want to test it:

{"id":"chatcmpl-redacted","object":"chat.completion.chunk","created":1697649987,"model":"gpt-3.5-turbo-0613","choices":{"0":{"index":0,"delta":{"role":"assistant","content":null,"function_call":{"name":"user_choice","arguments":""}},"finish_reason":null}}}

{"id":"chatcmpl-redacted","object":"chat.completion.chunk","created":1697649987,"model":"gpt-3.5-turbo-0613","choices":{"0":{"index":0,"delta":{"function_call":{"arguments":"{"}},"finish_reason":null}}}

{"id":"chatcmpl-redacted","object":"chat.completion.chunk","created":1697649987,"model":"gpt-3.5-turbo-0613","choices":{"0":{"index":0,"delta":{"function_call":{"arguments":" "}},"finish_reason":null}}}

{"id":"chatcmpl-redacted","object":"chat.completion.chunk","created":1697649987,"model":"gpt-3.5-turbo-0613","choices":{"0":{"index":0,"delta":{"function_call":{"arguments":" ""}},"finish_reason":null}}}

{"id":"chatcmpl-redacted","object":"chat.completion.chunk","created":1697649987,"model":"gpt-3.5-turbo-0613","choices":{"0":{"index":0,"delta":{"function_call":{"arguments":"}"}},"finish_reason":null}}}

{"id":"chatcmpl-redacted","object":"chat.completion.chunk","created":1697649987,"model":"gpt-3.5-turbo-0613","choices":{"0":{"index":0,"delta":{},"finish_reason":"stop"}}}

[lino-levan]

@userpixel Are you willing to make a PR? This looks good to me.