Update to latest Azure OpenAI Service preview API

[cecheta]

The following API versions will be retired on the 2nd April 2024:

2023-03-15-preview 2023-07-01-preview 2023-08-01-preview 2023-09-01-preview 2023-10-01-preview 2023-12-01-preview

We need to update all calls to Azure OpenAI to use the latest API version, 2024-02-01

• [x] #552
• [x] #562
• [x] #553

[ross-p-smith]

We are now on 2023-10-01-preview. #319

[ross-p-smith]

There is some extra info on citations here: #485

[andrewwiebe]

So we have taken this repo as a code base and have been customizing for our own internal app. I have been digging into these API changes a little bit. I was able to make some changes to the back end to reformat the response so the front end accepts the content, however the citations portion is significantly different and will require front end updates I think. A bit of information below, if it helps.

The citations used to come back in one long string with escaped double quotes in the messages array with the role tool as below.

"messages": [
                {
                    "delta": {
                        "role": "tool",
                        "content": "{\"citations\": [{\"content\": \"REDACTED", \"id\": null, \"title\": null, \"filepath\": null, \"url\": null, \"metadata\": {\"chunking\": \"orignal document size=650. Scores=4.7559686Org Highlight count=1.\"}, \"chunk_id\": \"0\"}], \"intent\": \"[\\\"What is BAM101?\\\", \\\"Tell me about BAM101\\\"]\"}"
                    },
                    "index": 0,
                    "end_turn": false
                }

There is no longer a messages array and citations comeback in split chunks in their own array with the role assistant as below.

"delta": {
                "role": "assistant",
                "context": {
                    "citations": [
                        {
                            "content": "REDACTED",
                            "title": null,
                            "url": null,
                            "filepath": null,
                            "chunk_id": "0"
                        },
                        {
                            "content": "REDACTED",
                            "title": null,
                            "url": null,
                            "filepath": null,
                            "chunk_id": "0"
                        },
                        {
                            "content": "REDACTED",
                            "title": null,
                            "url": null,
                            "filepath": null,
                            "chunk_id": "0"
                        },
                        {
                            "content": "REDACTED",
                            "title": null,
                            "url": null,
                            "filepath": null,
                            "chunk_id": "0"
                        },
                        {
                            "content": "REDACTED",
                            "title": null,
                            "url": null,
                            "filepath": null,
                            "chunk_id": "0"
                        }
                    ],
                    "intent": "[\"What is BAM101?\", \"Tell me about BAM101\"]"
                }
            },

Similar with the answer delta, the content is no longer in a messages array and is located directly in the choices array.

OLD

"choices": [
        {
            "index": 0,
            "messages": [
                {
                    "delta": {
                        "content": "B"
                    },
                    "index": 1,
                    "end_turn": false
                }
            ],
            "finish_reason": null
        }
    ]

NEW

"choices": [
        {
            "index": 0,
            "delta": {
                "content": "B"
            },
            "end_turn": false,
            "finish_reason": null
        }
    ]

And one last change I accounted for is that there is no longer a [DONE] message in the final response. We now need to check for "end_turn": true .

Looking forward to these updates so I can incorporate into our custom app. Let me know if I can help at all.

[andrewwiebe]

A couple other changes that are required for the outgoing call to the API.

The /extensions in the call at line below needs to be removed. It should just be /chat/completions.

https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/blob/a3e1ab5b02723989f1a3f2380f024478070bd459/code/app.py#L169

The body defining the the data source needs to be changed. Below line needs to change from "dataSources" to "data_sources"

https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/blob/a3e1ab5b02723989f1a3f2380f024478070bd459/code/app.py#L58

Below line needs to change from "AzureCognitiveSearch" to "azure_search"

https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/blob/a3e1ab5b02723989f1a3f2380f024478070bd459/code/app.py#L60

[ross-p-smith]

Hey @andrewwiebe - I'd gladly take a PR on this if you have time?

[andrewwiebe]

Hey @andrewwiebe - I'd gladly take a PR on this if you have time?

Unfortunately I wasn't able to get the changes fully functional. The changes with the delta and "answer" are pretty straight forward and can be reformatted by the back end for the front end to accept however the citations changes are a different animal. Front end changes are probably required to accept the new citations format, and react is definitely not my strong suit.

With some work we could probably update the backend to reformat the citations into the old format (which is what I was going to start looking into) but I am not so sure that is the best way forward. Probably better to re-code to the new openAI format as opposed to forcing the old format.

[cecheta]

Hello @andrewwiebe, thanks for all your help, we will try to update to 2024-02-01 and raise a pull request soon