Open minhduc0711 opened 1 month ago
And here I thought I was just holding it wrong...
I've been encountering the exact same issue. Would be very nice to see this fixed, as for our current use-case this is quite crucial, and with tool_choice=auto
, I'm getting a high rate of text responses that identify the tool to call correctly but don't actually trigger a tool call response.
I think the issue still is that the latest Azure OpenAI API with these options has not been released yet. It's confusing because the Azure client has the enum to set the latest version so we expect the service exist. Even some places of the docs and specs have the new version but the following page is still showing 2024-05-01-preview
. I think 2024-06-01
is what has this feature.
https://learn.microsoft.com/en-us/azure/ai-services/openai/api-version-deprecation
@janaka From what is shown on other pages, the 2024-06-01
version of the inference API has been released as GA. This page shows it as the current GA release: https://learn.microsoft.com/en-us/azure/ai-services/openai/reference . It was also mentioned on the "What's new page": https://learn.microsoft.com/en-us/azure/ai-services/openai/whats-new#new-ga-api-release. If it wasn't released yet, I would assume that you would get the same 404 error that you get if you submit a non-existent API version with a request.
But yeah, for some reason the current preview release is older than the current GA release.
@hobofan oh interesting. I'd not seen those pages before. If the GA bit was update when that page says it was last updated it's after I last tried. iirc I was getting a 404 or similar. Thanks for sharing those links.
Update: just tried it now. Still getting a 400 error. Invalid value: 'required'. Supported values are: 'none' and 'auto'.'
I wonder if there is a CLI command to check the API version for our deployment.
Looks like my assumption is invalid, that the Azure OpenAI API version numbers tracked the OpenAI OpenAI API version. They seem completely independent.
So Azure just have not released these features on the inference API.
So Azure just have not released these features on the inference API.
See the link to the specification in top-level comment. According the the spec, the required
option should now be supported (it didn't exist in the enum in previous versions). So either the spec is wrong, or this is a bug in the implementation.
ah I see, on main
. Totally not clear if main is release or what is the commit that's released as version X
@jpalvarezl @YunsongB @trrwilson any of you able to shed some light? Please help us understand what the actual released version of the inference API is? is the readme up to date? release_2023_05_15
is the released version? seems off to me.
Tried to do the same API calls, and looks like tool_choice="required"
is working now!
just tried now, still getting Invalid value: 'required'. Supported values are: 'none' and 'auto'.'
Maybe the update is still being rolled out.
Also getting Invalid value: 'required'. Supported values are: 'none' and 'auto'.'
with gpt-4o in east-us and sweden
Also getting Invalid value: 'required'. Supported values are: 'none' and 'auto'.
with gpt-4o in Canada.
This worked with gpt-3.5 turbo though.
Our deployment is east-us
@hobofan which region is your deployment?
From what I can tell in France Central (I only consume the deployment and don't have access to see the deployment details though).
Our deployments are also in France Central. Similar to what @vivia11 observed, gpt-3.5-turbo accepts required
, but 4o does not.
there's a new preview spec with tool_choice=required
that's something though prob means nothing given the stable spec has it but doesn't work in all regions.
gpt-3.5-turbo accepts
required
, but 4o does not.
Yeah, this still happens in 2024-07-01-preview
for me.
gtk - how are you testing? curl or something? at least yesterday the dotnet lib didn't have that version in the enum.
I used the Python script in https://github.com/Azure/azure-rest-api-specs/issues/29844#issue-2410566860 and simply updated the api_version
parameter.
I used the Python script in #29844 (comment) and simply updated the
api_version
parameter.
gotcha - with gtp-4o right?
Yes with gpt-4o
it will be interesting to see what happens when they release gpt-4o 2024-08-06
on the inferencing API as that's a bit useless without an updated REST API.
So 2024-07-01-preview
announced on the what's new page but the tool_choice=required isn't deployed yet :( https://learn.microsoft.com/en-us/azure/ai-services/openai/whats-new
API Spec link
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/stable/2024-06-01/inference.yaml#L1678-L1686
API Spec version
2024-06-01
Describe the bug
In the specification file above, I see that the chat completion's parameter
tool_choice
should supportrequired
as a valid value, but I got an error when setting it as such.Expected behavior
I should be able to set
tool_choice="required"
, and the endpoint will always return a tool call.Actual behavior
I got the following error:
Reproduction Steps
Environment
openai python package version: 1.35.14