A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
Add an option to use models deployed on HuggingFace and available through its serverless inference API. The new approach supports streaming, thus there is no visible difference between HF and OpenAI models.
Wrappers were made to create a general way to support multiple LLM clients, including HF Inference, openai and possibly more in the future. By changing the chat_completion method, we are able to control how the completion is done (e.g. multimodel approach).
Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.
[ ] Yes
[x] No
Does this require changes to learn.microsoft.com docs?
This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.
[ ] Yes
[x] No
Type of change
[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:
Purpose
Add an option to use models deployed on HuggingFace and available through its serverless inference API. The new approach supports streaming, thus there is no visible difference between HF and OpenAI models. Wrappers were made to create a general way to support multiple LLM clients, including HF Inference, openai and possibly more in the future. By changing the chat_completion method, we are able to control how the completion is done (e.g. multimodel approach).
Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error? If you're not sure, try it out on an old environment.
Does this require changes to learn.microsoft.com docs?
This repository is referenced by this tutorial which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial, check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.
Type of change
Code quality checklist
See CONTRIBUTING.md for more details.
python -m pytest).python -m pytest --covto verify 100% coverage of added linespython -m mypyto check for type errorsruffandblackmanually on my code.