stream resposne contains a null tool_calls , invalid for typescript resposne validation

[yangcheng]

Describe the bug

tool_calls should be an empty array, not missing. Should not be null. The null is causing typescript type validation error.

{
    "id": "chatcmpl-bc23d19d",
    "created": 1717383192,
    "model": "meta.llama3-70b-instruct-v1:0",
    "system_fingerprint": "fp",
    "choices": [
        {
            "index": 0,
            "finish_reason": "stop",
            "logprobs": null,
            "delta": {
                "role": "assistant",
                "content": "",
                "tool_calls": null
            }
        }
    ],
    "object": "chat.completion.chunk",
    "usage": null
}

In OpenAI's official sdk, tool_calls can be undefined or empty_array, but not null.

The type is defined here https://github.com/openai/openai-node/blob/fd70373450d6c39ff55d984a2ff13ea7a7df23d1/src/resources/chat/completions.ts#L434

export namespace Choice {
    /**
     * A chat completion delta generated by streamed model responses.
     */
    export interface Delta {
      /**
       * The contents of the chunk message.
       */
      content?: string | null;

      /**
       * @deprecated: Deprecated and replaced by `tool_calls`. The name and arguments of
       * a function that should be called, as generated by the model.
       */
      function_call?: Delta.FunctionCall;

      /**
       * The role of the author of this message.
       */
      role?: 'system' | 'user' | 'assistant' | 'tool';

      tool_calls?: Array<Delta.ToolCall>;
    }

Please complete the following information:

• [x] Which API you used: /chat/completions
• [x] Which model you used: meta.llama3-70b-instruct-v1:0

To Reproduce Steps to reproduce the behavior. If possible, please share an example request.

Expected behavior I understand that python allows null and undefined to be None. A more acceptable behavior will be omit the value in response if model does not support tool_calls or some optional field

Screenshots If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).

Additional context Add any other context about the problem here.

[yangcheng]

for non stream mode, the response from llama does not contain tool_calls , so just remove tool_calls from stream response chunk would make the API more consistent, and more compatible with OpenAI

{
    "id": "chatcmpl-602caf75",
    "created": 1717387517,
    "model": "meta.llama3-70b-instruct-v1:0",
    "system_fingerprint": "fp",
    "choices": [
        {
            "index": 0,
            "finish_reason": "stop",
            "message": {
                "role": "assistant",
                "content": "**masked content ***"
            }
        }
    ],
    "object": "chat.completion",
    "usage": {
        "prompt_tokens": 477,
        "completion_tokens": 165,
        "total_tokens": 642
    }
}

[daixba]

This should be fixed, please try it again.

[daixba]

Close for no updates