search

RageAgainstThePixel / OpenAI-DotNet

A Non-Official OpenAI RESTful API Client for DotNet
https://openai.com
MIT License
699 stars 149 forks source link

Default API version 2022-12-01 results in 404 on chat completion request for new Azure OpenAI deployments #112

Closed sibbl closed 1 year ago

sibbl commented 1 year ago

Bug Report

Overview

By default, "2022-12-01" is used as the ApiVersion. However, this results in a 404 being returned from Azure OpenAI when the deployment was recently created. With the same setup and only setting ApiVersion to "2023-05-15", a valid result is returned.

{
    "error": {
        "code": "404",
        "message": "Resource not found"
    }
}

To Reproduce

Steps to reproduce the behavior:

  1. Create a new OpenAIClient with only resourceName and deploymentId from Azure being set.
  2. Use the api.ChatEndpoint.GetCompletionAsync method to make a completion request

Expected behavior

It should work out of the box.

Screenshots

image image

Additional context

For the first basic tests, updating the ApiVersion seems to be no problem. However, in a quick search I could not find a diff between both API versions. Maybe this issue could be migrated into a generic "support API version 2023-05-15" feature support ticket?

StephenHodgson commented 1 year ago

So the default API version is no longer valid? Should be an easy update.

Would you mind making the necessary changes?

StephenHodgson commented 1 year ago

I am a bit concerned about the fact that Azure is giving a 404. I wonder why they don't keep it when all the docs still consider it "stable"

StephenHodgson commented 1 year ago

I'm wondering if Azure is defaulting to the latest API version (which idk why they don't just actually version it with semver instead of this datetime crap)

ADefWebserver commented 1 year ago

My understanding is deploymentId is specific to each Azure Tenant. There is no "Global Deployment Id". This is why you have to set this youself:

or am I misunderstanding the issue?

StephenHodgson commented 1 year ago

This is why you have to set this youself

This is what I mean. If you choose a different API version, then it would make sense you would get a 404 if you also don't update the client to use the specified version.

ADefWebserver commented 1 year ago

This is why you have to set this youself

This is what I mean. If you choose a different API version, then it would make sense you would get a 404 if you also don't update the client to use the specified version.

So, there is no Bug, correct?

StephenHodgson commented 1 year ago

I don't think so, but it prob doesn't hurt to bump the default API version

ADefWebserver commented 1 year ago

If someone uses the "default value" it's gonna eventually break when they update OpenAI-DotNet but their model in Azure is still an older version. (again, let me know if I am simply misunderstanding the issue!)

StephenHodgson commented 1 year ago

Hmm this is a good point

ADefWebserver commented 1 year ago

(also Syncfusion just finished tech edits on my book, expect a ton of traffic once it gets released because all examples use OpenAI-DotNet 😉)

StephenHodgson commented 1 year ago

@sibbl I kinda agree with @ADefWebserver. This may be a breaking change. You should update your client settings to specify the api version you are targeting.

sibbl commented 1 year ago

@ADefWebserver @StephenHodgson I also thought about this before committing, but as far as I understand the MS docs, older deployments will also be available out of the box with the new API version. They just don't deploy new deployments under the old API version.

I'm referring to their "What's new in Azure OpenAI" page.

If you are currently using the 2023-03-15-preview API, we recommend migrating to the GA 2023-05-15 API. If you are currently using API version 2022-12-01 this API remains GA, but does not include the latest Chat Completion capabilities.

Maybe somebody with older deployments on Azure can confirm this?

Technically speaking, I compared the Swagger docs of the Azure APIs (see the links to them in my PR) and the old deployments shouldn't even have support for chat completion at all... so basically everything is still in preview anyway.

ADefWebserver commented 1 year ago

@sibbl - My "hope" is that what is required is to:

  1. Create a new Deployment in Azure OpenAI
  2. Update the DeploymentIdin the connection settings for OpenAI-DotNet
  3. Everything should work?
StephenHodgson commented 1 year ago

yeah I think the real fix here is just to make sure you specify your apiVersion when setting up the client:

var auth = new OpenAIAuthentication("sk-apiKey");
var settings = new OpenAIClientSettings(resourceName: "your-resource-name", deploymentId: "deployment-id", apiVersion: "2023-03-15-preview");
var api = new OpenAIClient(auth, settings);
StephenHodgson commented 1 year ago

I compared the Swagger docs of the Azure APIs

Now we do likely need to update the api to utilize some of the new changes in the schema. I did do a diff myself just to see. Only really interesting thing is the chat extensions (likely similar to the OpenAI plugins), but I'm unsure how much I want to try and support that. Might be better off just using the official Azure OpenAI Client, tbh.

I started this project over 2 years ago, long before the Azure OpenAI offerings.

sibbl commented 1 year ago

@StephenHodgson as mentioned in my PR, the only diff between both spec files is the addition of the chat completion endpoint and the version change. For me, the endpoint is working as expected when using it with the latest Nuget version of this project.

Also, I think that either your option (recreate deployment) is a solution, or it works out of the box because Microsoft deployed the new version also for old deployments. I see no reason why they wouldn't do this and also understand their "What's new" hint box in that way.

But yes, I agree that specifying the API version might be the best solution for now. Maybe the Readme could aggregate supported API versions.

However, as the whole Microsoft Azure OpenAI is still in preview stage, I personally would kind of expect my application to break when updating dependencies anyway. But that's just my personal opinion.

StephenHodgson commented 1 year ago

No worries, I appreciate your support for my project!

If you'd like to update any support for future Azure offerings, please go ahead and submit a PR. I do not have access so cannot verify for myself unfortunately

ADefWebserver commented 1 year ago

Might be better off just using the official Azure OpenAI Client, tbh

The official package is way behind and does not offer nearly the features that this one does. I predict that the future will have 50% of the people using this package, using it on Azure OpenAI.

ADefWebserver commented 1 year ago

or it works out of the box because Microsoft deployed the new version also for old deployments.

I 'think' (I could be wrong) that if we don't use this option that wont happen?

image

sibbl commented 1 year ago

@ADefWebserver this drop-down refers to the model version - not the API version. Like when a new model version of gpt-3.5 will be released, you can update from the 0301 to the new one using this dropdown.

Afaik one cannot see or control the API version anywhere in the UI.

But do you have a chat completion deployment from April or earlier and can test whether the 2023-05-15 version also works out of the box for this older deployment?

StephenHodgson commented 1 year ago

I don't like how different and difficult MSFT has made it tbh. I don't understand why the API surface is so different from the OpenAI offering.

ADefWebserver commented 1 year ago

But do you have a chat completion deployment from April or earlier and can test whether the 2023-05-15 version also works out of the box for this older deployment?

Yeah I see your point.

I have a deployment:

https://{my-resource-name}.openai.azure.com/openai/deployments/GPT35Turbo-0301/completions?api-version=2022-12-01

I use these values:

    "ApiKey": "{my API Key}"
    "ResourceName": "{my-resource-name}"
    "DeploymentId": "GPT35Turbo-0301"
    "ApiVersion": "2022-12-01"

I update to the latest OpenAI-DotNet and I get:

GetCompletionAsync Failed! HTTP status code: NotFound | Response body: {"error":{"code":"404","message": "Resource not found"}}

ADefWebserver commented 1 year ago

However if I go to the Chat section in Azure OpenAI (as opposed to the Completions section):

image

I see this deployment URL:

https://{my-resource-name}.openai.azure.com/openai/deployments/GPT35Turbo-0301/chat/completions?api-version=2023-03-15-preview

I use these values:

   "ApiKey": "{my API Key}"
    "ResourceName": "{my-resource-name}"
    "DeploymentId": "GPT35Turbo-0301"
    "ApiVersion": "2023-03-15-preview"

I update to the latest OpenAI-DotNet and it works!

ADefWebserver commented 1 year ago

@sibbl I think that api-version=2022-12-01 that only shows up on the Completions (/completions) section in Azure OpenAI not the Chat (/chat/completions) section, is invalid when calling the Chat API?

I think this is what the issue is?

ADefWebserver commented 1 year ago

I don't like how different and difficult MSFT has made it tbh. I don't understand why the API surface is so different from the OpenAI offering.

I actually consider this a feature 😎

Microsoft Azure OpenAI allows you to have your own deployments and manage the quota and scaling per deployment. I think this is what is required for true enterprise deployments. OpenAI's offering is-not-that.

(of course I am sure there is 'shared' resources at some point because that is how the cloud works...)

StephenHodgson commented 1 year ago

I just hate that the API surface isn't smooth enough for this type of missmatch

ADefWebserver commented 1 year ago

calling GPT-3.5-turbo is much faster on an Azure Open AI deployment...

Azure Open AI is 3.30 times faster than OpenAI

StephenHodgson commented 1 year ago

Azure Open AI is 3.30 times faster than OpenAI

I'm well aware

sibbl commented 1 year ago

Some interesting things:

I think we have two options here:

a) make the API version for Azure credential setup a mandatory field. We might want to consider offering an Enum in the future.

b) go with the latest stable API version and hope for MS not to update the API a lot, and also not deprecate older versions too often (so that no 404 occurs, and also newer API versions are deployed automatically).

It would be helpful to have some insights from Microsoft on this topic. Without knowing this, perhaps it would be the best to document this now (like add a hint to the Azure setup section of this repo's readme). Then let's wait and observe how often Microsoft changes APIs (I don't expect any changes without OpenAI adding new stuff to their API) and how the deal with backwards compatibility (more 404s or not).

ADefWebserver commented 1 year ago

@sibbl I agree with everything you said...

I think we have two options here:

a) make the API version for Azure credential setup a mandatory field. We might want to consider offering an Enum in the future.

b) go with the latest stable API version and hope for MS not to update the API a lot, and also not deprecate older versions too often (so that no 404 occurs, and also newer API versions are deployed automatically).

I vote for option A :)

Thanks for the links. Also, I found this:

Working with the ChatGPT models

image

And this:

API Key:

image