search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
5.94k stars 4.08k forks source link

Possibility of chat replying using both PDF documents and information available in the model knowledge base. #1803

Closed FranciscoMMendes closed 2 months ago

FranciscoMMendes commented 2 months ago

Please provide us with the following information:

I would like to know if it is possible to have a chatbot that can reply to questions using the PDFs provided but also to use the model's own knowledge base to answer any other questions, like chatgpt does. So it would be some kind of a hybrid version of a RAG.

Is this possible using this current solution? Would it just be a case of changing the prompt to be able to reply to both knowledge bases?

Thank you in advance for your input.

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here.

Versions

Mention any other details that might be useful

Thanks! We'll be in touch soon.

pamelafox commented 2 months ago

That should already be possible, you would want to change the prompt to specify that it does not have to answer according to sources. The prompt is here: https://github.com/Azure-Samples/azure-search-openai-demo/blob/e8fd8190151f9af6e8b38670c5ddd84a0e793fae/app/backend/approaches/chatreadretrieveread.py#L57

Where things can get interesting is when a user asks a question that should have an answer from the PDF documents, but they get a general answer instead. For example, if they ask "what does a PM do?" and your documents have their own definition of "PM", but the model answers with what it knows generally about PMs. So I would recommend running evaluations on questions in the overlap to make sure you're happy with the response.