[BUG] `tool_choice="required"` is still not supported in the latest OpenAI API version

minhduc0711 commented 1 month ago

API Spec link

https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/stable/2024-06-01/inference.yaml#L1678-L1686

API Spec version

2024-06-01

Describe the bug

In the specification file above, I see that the chat completion's parameter tool_choice should support required as a valid value, but I got an error when setting it as such.

Expected behavior

I should be able to set tool_choice="required", and the endpoint will always return a tool call.

Actual behavior

I got the following error:

Traceback (most recent call last):
  File "/data/mdm/iva_webapi/test_openai.py", line 16, in <module>
    chat_completion = client.chat.completions.create(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mdm/miniforge3/envs/iva/lib/python3.11/site-packages/openai/_utils/_utils.py", line 277, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/data/mdm/miniforge3/envs/iva/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 643, in create
    return self._post(
           ^^^^^^^^^^^
  File "/data/mdm/miniforge3/envs/iva/lib/python3.11/site-packages/openai/_base_client.py", line 1250, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mdm/miniforge3/envs/iva/lib/python3.11/site-packages/openai/_base_client.py", line 931, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/data/mdm/miniforge3/envs/iva/lib/python3.11/site-packages/openai/_base_client.py", line 1030, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "'$.tool_choice' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}

Reproduction Steps

import os

from openai import AzureOpenAI

gpt_35_16k_endpoint = os.getenv("OPENAI_GPT_35_16K_ENDPOINT")
gpt_35_16k_key = os.getenv("OPENAI_GPT_35_16K_KEY")
gpt_35_16k_deployment = os.getenv("OPENAI_GPT_35_16K_DEPLOYMENT")

if gpt_35_16k_deployment and gpt_35_16k_key and gpt_35_16k_endpoint:
    client = AzureOpenAI(
        azure_endpoint=gpt_35_16k_endpoint,
        api_key=gpt_35_16k_key,
        api_version="2024-06-01",
    )

    chat_completion = client.chat.completions.create(
        model=gpt_35_16k_deployment,
        messages=[
            {
                "role": "user",
                "content": "Weather in hanoi today",
            }
        ],
        tools=[
            {
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "description": "Get the current weather in a given location",
                    "parameters": {                                                                                    
                        "type": "object",                                                                              
                        "properties": {                                                                                
                            "location": {"type": "string", "description": "The city and state, e.g. San Francisco, CA"}
                        },
                    },
                },
            }
        ],
        tool_choice="required",
    )
    print(chat_completion.to_json(indent=2))

Environment

openai python package version: 1.35.14

hobofan commented 1 month ago

And here I thought I was just holding it wrong...

I've been encountering the exact same issue. Would be very nice to see this fixed, as for our current use-case this is quite crucial, and with tool_choice=auto, I'm getting a high rate of text responses that identify the tool to call correctly but don't actually trigger a tool call response.