[Bug]: generatePlan will not create chat request with user input when using AzureOpenAI

[adashen]

Please note: any submissions with insufficient reproducible information will be marked as 'Waiting for customer input' and may be closed is there is no response

Describe the bug

Because of https://github.com/microsoft/teams-ai/blob/main/js/packages/teams-ai/src/OpenAIPlanner.ts#L177 , the chat request could not be created correctly when using Azure Open AI deploy name.

To Reproduce

Steps to reproduce the behavior:

1. Deploy a "gpt-35-turbo" model in Azure Open AI with deploy name gpt35.
2. Open "04.ai.d.chainedActions.listBot" sample in VSCode with teams toolkit installed.
3. Update https://github.com/microsoft/teams-ai/blob/main/js/samples/04.ai.d.chainedActions.listBot/src/prompts/chatGPT/config.json#L13 with deployment name "gpt35"
4. Update the source code to use AzureOpenAIPlanner
5. Start to run the bot
6. After installation send "I want to have a shopping list" to the listBot.
7. Bot return "I'm not sure I'm the best person to help with is."

Expected behavior

The bot should run createList action and say "List Shopping has been created."

=========== Admins area:

• [x] Discussed with Steve; we want to iterate on SK changes.
• [ ] @singhk97 to explore SK changes
• [ ] @corinagum add named model objects

[corinagum]

Hi @adashen, could you verify that the deployment name 'gpt-3.5-turbo' is being used? Your step 1 does not have the '.'

[adashen]

Hi @adashen, could you verify that the deployment name 'gpt-3.5-turbo' is being used? Your step 1 does not have the '.'

The model name in Azure Open AI is "gpt-35-turbo". The model name is OK. But the problem is in SDK when using Azure Open AI, the "deployment name" is used to connect to the model, not the model name. So if the deployment name is not prefix with "gpt-", it will cause problem. And my workaround is to have a new deployment name "gpt-35-turbo" for the model.

In the above image, there are two deployments for model "gpt-35-turbo". However the "testgpt" deployment cannot work. I can only use "gpt-35-urbo" deployment.

[corinagum]

Hi @adashen, thanks for the clarification, that helped me a lot :)

You're right, it looks like deployment name is asked for in order to set up the API calls, but I don't think we can avoid having the user provide model name in order to properly determine chat or text completion. I'm bringing this up for discussion with my team and we'll get working on a fix asap!

---

As noted above, for now the workaround for this is to give the Azure OpenAI deployment a name that starts with gpt if you are using a gpt model (chat completion).

[garrytrinder]

Just ran into this issue.

Currently the type of request is determined by the model name.

https://github.com/microsoft/teams-ai/blob/040282ee992e909e087228136bae55eb5bb867c3/js/packages/teams-ai/src/OpenAIPlanner.ts#L136-L156

This is fine for OpenAI, but when using Azure OpenAI Service, the deployment name is used instead of the model name, which can be any value.

We already have a type property defined in config.json and in many samples this is set to completion.

https://github.com/microsoft/teams-ai/blob/040282ee992e909e087228136bae55eb5bb867c3/js/samples/04.ai.a.teamsChefBot/src/prompts/chat/config.json#L1-L16

It would be nice to be able to change the type property to chat to determine the request made by the SDK rather than relying on the model name set in the planner.

[singhk97]

@adashen For now you should rename the AOAI model to start with gpt- for it to work. I have kickstarted the process to make the right changes here: https://github.com/microsoft/teams-ai/issues/491.

@garrytrinder text completion api don't support gpt- models and chat completion api don't support text- models. With your proposal it's possible to configure an api to the wrong model, which should be avoided.

Feel free to add your thoughts to the feature request to fix this issue: https://github.com/microsoft/teams-ai/issues/491.

[corinagum]

AI behavior has undergone a major update and includes a fix for this scenario. #721