Azure-Samples / chat-with-your-data-solution-accelerator

A Solution Accelerator for the RAG pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences. This includes most common requirements and best practices.
https://azure.microsoft.com/products/search
MIT License
656 stars 327 forks source link

Include image citations in prompt/response #964

Open cecheta opened 1 month ago

cecheta commented 1 month ago

Required by #323

Description

When passing images to GPT-4 vision in the question answer tool, the prompt should be modified if advanced image processing is enabled, so that the LLM is instructed to include citations to the image in the response, and the response is parsed as appropriate.

Tasks

_To be filled in by the engineer picking up the subtask