search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
5.88k stars 4.02k forks source link

"text-davinci-003" Model is unavailable to use with **new deployments.** since Friday 7th July 2023 #406

Closed yeggan closed 1 year ago

yeggan commented 1 year ago

MS announced on 7July 2023, some older models in Azure OpenAI Service will be retired on 5 July 2024. One of the "Base models to be retired" is "text-davinci-003" and "Anticipated replacement model" is "gpt-35-turbo-instruct". Then MS mentioned Beginning today, the models being retired will be unavailable to use with new deployments.** So, when tried to use this repo for new deployment I received error. Then I changed "text-davinci-003" to "gpt-35-turbo-instruct" model and received below error.

Deployment Error Details: InvalidTemplateDeployment: The template deployment 'openai' is not valid according to the validation procedure. The tracking id is 'xxxxxxxxxxxxxx'. See inner errors for details. DeploymentModelNotSupported: The model 'Format: OpenAI, Name: gpt-35-turbo-instruct, Version: 1, Source: ' of account deployment is not supported.

Could you please let us know how to use this repo for new deployment??

Python3-Pyro commented 1 year ago

I think this issue is already in discussion with no right answer - https://github.com/Azure-Samples/azure-search-openai-demo/issues/388. Guess we will have to test it out ourselves to figure it out.

johnnyreilly commented 1 year ago

Reference to announcement: https://techcommunity.microsoft.com/t5/ai-cognitive-services-blog/announcing-updates-to-azure-openai-service-models/ba-p/3866757

Retiring older models

In parallel with these announcements, starting today we will retire the older instruct model (‘text-[series]-00x’) and fine-tunable models (Ada, Babbage, Curie and Davinci). They will be unavailable for new customers and they will be retired for existing customers on July 5, 2024.

I think this is the relevant announcement

TroyHostetter commented 1 year ago

We had the same thing. You need to either merge prior branches or manually make some tweaks to main.bicep and cognitiveservices.bicep. Have a look here https://github.com/Azure-Samples/azure-search-openai-demo/issues/313

Microsoft removed all-things davinci as well as most of the other models, there's only 3 now.

TroyHostetter commented 1 year ago

Also, be warned, if you are deleting your Resource Group, you will not be able to recover Davinci. I had been deleting the Resource Group on my MSDN subscription, hoping I could "shelf" and redeploy when my time freed up to look at it again. I could not recover Davinci, it was gone for good. So, be warned.

pamelafox commented 1 year ago

This has since been resolved, we updated the repo with some prompt tweaks and few-shot examples that get gpt35-turbo working just as well as the old davinci model.

kushalbhabra commented 1 year ago

Hi @pamelafox appreciate if you can point to the PR or commit that has these prompt tweaks and few-shot examples for comparison.

pamelafox commented 1 year ago

Here's the previous code that queried davinci, with the Completions API: https://github.com/Azure-Samples/azure-search-openai-demo/blob/ef32ced17592ef47ba7da576dc1b93bdee82da09/app/backend/approaches/chatreadretrieveread.py

And here's the latest code: https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/chatreadretrieveread.py

The prompt itself is not very different (https://www.diffchecker.com/vzVhRZAc/) but the new approach also uses the ChatCompletions API and includes some few-shot examples of generated queries.