For AzureOpenAI, may need another depolyment id?

[oliu321]

I am using Azure OpenAI, and I have no problem with the Visual chatGPT studio, but when I am trying to use turbo chat, the API returns 404 resource not found. I noticed that I can only have one model for each deployment in the Azure AI Studio (the depolyment ID I put in Visual Studio Options uses TextDavinci003), so may be I need have another deploymentId for GPT_3_5_Turbo? (If so maybe there will have to be two API instances - one for main and one for turbo chat?) Anyway, seems a difference between AzureOpenAI and OpenAI.

[jeffdapaz]

Ok, I understand the problem, however I can't test it myself since I don't have access to Azure OpenAI yet.

But from what I've noticed, it really seems to me that a different deployment is needed for Turbo Chat. Or it can be related with the API Version.

Can you perform this test, for example by changing your deployment model to gpt-3.5-turbo or creating a new deployment with this model and testing to see if it works?

@Rabosa616 , because you showed to me how to implement the connection to azure, how have you been using the tool? Did you have a problem similar to @oliu321 ?

[oliu321]

@jeffdapaz Ok, I just did some test so here is what I found:

1. I am not 100% sure whether you will need a different deployment for Turbo Chat, but probably it's the way to go because in the Azure OpenAi, one deployment can only use one model (and it's a bijection so one model can only be used in one deployment - which makes little sense to me but it's what it is)

2. I saw a model which is called code-davinci-002 in the model list, may be use that for the coding (but keep the gpt-35-turbo for turbo chat?)

3. I created a deployment just for gpt-35-turbo model, now there is no more 404 but the URL in the calls to Azure OpenAI was wrong so it will still fail: Azure expects: https://[Azure Open Ai Resource Name].openai.azure.com/openai/deployments/gpt35turbofordev/completions?api-version=2022-12-01 , but your extension is using (an extra chat): https://[Azure Open Ai Resource Name].openai.azure.com/openai/deployments/gpt35turbofordev/chat/completions?api-version=2022-12-01

4. It seems if I change values in the tools/options (for example change the Azure Deployment ID) I will have to restart visual studio for new values to be loaded - seems it was cached somewhere and cached value would be used without a restart of visual studio

[oliu321]

Another issue related to this, now I set the deployment gpt35turbofordev (which is my name for gpt-35-turbo model) but I tried to use the chat functionality (not the turbo one) actually the call to the Azure OpenAPI seems to be also working but: if I put a string "test" in the request box, then response will be an ever increasing infinite (seems to be) {"test", "test", "test", ...} and the extension will hang and the whole visual studio would be frozen.

[Rabosa616]

@oliu321 & @jeffdapaz I will check it next week, this week I have no capacity at all, sorry for that.

[jeffdapaz]

@oliu321 , follow my answers:

1 - Probably, but it won't be just that. See what I say in "3.

2 - OpenAI discontinued this model, so I recently removed it from the extension options. I believe it will soon be removed from Azure as well.

In my opinion for coding the best template is TextDavinci003. And yes, for chat Turbo use the gpt-3.5-turbo, or GPT-4 if you prefer.

3 - I believe that now the problem is related to what I had said about the API version.

I'm using a library to communicate with OpenAI/Azure, and in its repository I found this issue below. Read the last comment:

https://github.com/OkGoDoIt/OpenAI-API-dotnet/issues/81

Can you confirm wich version your deployment are using? Maybe I need add in the option the deployment specific for the Turbo Chat and another options to set the API version for Azure.

4 - It's because I can't get when the options was changed. It's only will be changed when create a new "chatGPT object" instance. Maybe close the chat window and open again do the trick.

About your test on other tool window, I need to see. When we make a generic request, OpenAI responds with random responses, however with me blocking VS never happened.

@Rabosa616 , no problem. Very thanks!

[oliu321]

3. API version I was using 2022-12-01 which was suggested by Azure (they generates sample code for you) I didn't see a way to specify API version in the Azure AI Studio, but if you want to add new version for turbo chat like 2023-03-15-preview
4. I will check more and let you know. Thanks.

[jeffdapaz]

@oliu321,

If I prepare a version with these changes and share the installer here, can you install and test if our suggestions will solve the problem?

If they solve, I can publish it on the marketplace for everyone, knowing that it is the "final solution" for Azure.

[oliu321]

@jeffdapaz Yes I can and thanks.

[jeffdapaz]

@oliu321 ,

ok, I will prepare these changes.

[jeffdapaz]

@oliu321 ,

Here you can get the VSIX file with my modifications.

You will see the new options "Azure Turbo Chat Deployment ID" and "Azure Turbo Chat API Version".

The API version option is not required, so you can test with and without setting the version to see how works.

Tell me something later.

[oliu321]

@jeffdapaz Just tested it, I will have to set both "Azure Turbo Chat API Version" to 2023-03-15-preview and "Azure Turbo Chat Deployment ID" to the gpt35 deployment and then it works. Setting "Azure Turbo Chat API Version" to 2022-12-01 or leaving it blank won't work.

Thanks, good enough for me for now.

[jeffdapaz]

@oliu321,

Thanks for the feedback!

I will publish on marketplace a new version with these changes.