[Feature Request]: Moving away from completions api

[singhk97]

Abstract

OpenAI has publicly stated they are moving away from the Completions API and will not be developing it any further. It has been replaced by Chat Completions API, which was released in March 2023. It reportedly accounts for 97% of the API usage.

Overview

As of July 6 2023, OpenAI has rendered the aCompletion API legacy API. The API only supports are text-davinci-003 models or older. To utilize gpt-3.5-turbo or gpt-4, you must use the Chat Completion API. Here's the blog post: https://openai.com/blog/gpt-4-api-general-availability.

You can only use the text-davinci-003 model with the Completions API. And the gpt- models only with the Chat Completions API. This applies to both OpenAI and Azure OpenAI.

Starting January 4, 2024, older completion models will no longer be available, and will be replaced with the following models:

Essentially, the Chat Completions API is a structured interface built upon the Completions API. This implies that you can use the Chat Completions API to function similarly, by omitting conversation history.

Proposal

• [ ] 1. Replace all the samples with gpt-3.5-turbo, and move away from text-davinci-003.
• [ ] 2. Set Chat Completions API as the default completion api, and use Completions API only if the model is prefixed with text-.
  • PROs: AOAI deployment names do not need to be prefixed with gpt- for the planner to utilize the Chat Completion API. This is something the community would like: https://github.com/microsoft/teams-ai/issues/394
  • CONs: Older models not prefixed with text- will not be usable. We can eventually add support for babbage-002 and davinci-002 if the users would like so.

[corinagum]

This looks good to me -- I like the idea of focusing towards chat completion. In the changes I'm currently making, instead of parsing the model name, I added an option to indicate completion type:

(Please ignore the class naming -- that is temporary)

So, combining with what is above, this is what I'm thinking

1. Add an option when creating the planner to indicate completionType as chat (default) or completion (text)
2. Continue to take model name (needed for OpenAI API calls)
3. For Azure OpenAI, require deployment name param (required for API calls) instead of the model name param
   1. Add inline documentation so that customers know model name and deployment name are separate
4. Update samples to use chat completion

The benefit is that no model name parsing will need to be done - instead it is up to the user to 'opt in' to text completion. It will also be forward thinking like @singhk97's proposal where we're preparing for text completion to go away.

Thoughts? @singhk97 @aacebo

[aacebo]

This looks good to me -- I like the idea of focusing towards chat completion. In the changes I'm currently making, instead of parsing the model name, I added an option to indicate completion type:

(Please ignore the class naming -- that is temporary)

So, combining with what is above, this is what I'm thinking

1. Add an option when creating the planner to indicate completionType as chat (default) or completion (text)
2. Continue to take model name (needed for OpenAI API calls)

3. For Azure OpenAI, require deployment name param (required for API calls) instead of the model name param

   1. Add inline documentation so that customers know model name and deployment name are separate
4. Update samples to use chat completion

The benefit is that no model name parsing will need to be done - instead it is up to the user to 'opt in' to text completion. It will also be forward thinking like @singhk97's proposal where we're preparing for text completion to go away.

Thoughts? @singhk97 @aacebo

I agree, but we might want to tweak the naming since a chat in openai is a completion still. The change they made was Completion => Chat Completion, so this may confuse devs. might consider 'chat' | 'text'

[singhk97]

Agreed @aacebo

I like the idea @corinagum. If there is an option to specify the completions api to use, then we won't need the AOAI model name anymore. In that case we should just keep one property to capture model name and deployment name.

Here's a thought, what if we changed the modelName to deploymentName property. AOAI customers won't be confused because the name matches what they should provide. And we can add a comment for OpenAI customers to supply the model name. Or having it the other way around also works for me as well.

[corinagum]

With Steve's breaking changes merged in, I think we can mark this as closed. Priority has been placed on chat completion over text completion, and further AI work for C# is being done by Qianhao.