A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
The AI Chat Protocol has now been further codified in an SDK (https://github.com/microsoft/ai-chat-protocol). We are not yet porting to the SDK due to our need for authentication and some questions around the token provider option in the SDK. However, we'd still like for our HTTP API to comply to the Chat Protocol. There are two differences from our current API design:
Instead of sending "stream": True to get a streaming response, we send to a different path, "/chat/stream". That is more compatible with API documentation generators like Swagger/TypeSpec, as they currently don't have a way of specifying a completely different response type based off a parameter inside a JSON, but they definitely can vary response type based on path. It's also potentially better for monitoring when it's clear from the path that you're making a streaming response. It's actually what @srbalakr originally had in this repo, and we only changed it to match the original proposed version of the protocol. You live, you learn!
Instead of passing down the full choices list from the OpenAI response, we pass down only the first choice, specifically it's message or delta. That's because we never use the other choices in the frontend, and we also don't use the additional OpenAI metadata in the frontend. So it simplifies our frontend code and reduces the size of responses sent to the browser.
Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.
[ ] Yes
[X] No
Does this require changes to learn.microsoft.com docs?
This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.
[ ] Yes
[X] No
Type of change
[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:
Purpose
The AI Chat Protocol has now been further codified in an SDK (https://github.com/microsoft/ai-chat-protocol). We are not yet porting to the SDK due to our need for authentication and some questions around the token provider option in the SDK. However, we'd still like for our HTTP API to comply to the Chat Protocol. There are two differences from our current API design:
"stream": True
to get a streaming response, we send to a different path, "/chat/stream". That is more compatible with API documentation generators like Swagger/TypeSpec, as they currently don't have a way of specifying a completely different response type based off a parameter inside a JSON, but they definitely can vary response type based on path. It's also potentially better for monitoring when it's clear from the path that you're making a streaming response. It's actually what @srbalakr originally had in this repo, and we only changed it to match the original proposed version of the protocol. You live, you learn!choices
list from the OpenAI response, we pass down only the first choice, specifically it's message or delta. That's because we never use the other choices in the frontend, and we also don't use the additional OpenAI metadata in the frontend. So it simplifies our frontend code and reduces the size of responses sent to the browser.Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error? If you're not sure, try it out on an old environment.
Does this require changes to learn.microsoft.com docs?
This repository is referenced by this tutorial which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial, check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.
Type of change
Code quality checklist
See CONTRIBUTING.md for more details.
python -m pytest
).python -m pytest --cov
to verify 100% coverage of added linespython -m mypy
to check for type errorsruff
andblack
manually on my code.