search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
6.31k stars 4.22k forks source link

Ensure that GPT never answers from outside its sources #564

Open viru2206 opened 1 year ago

viru2206 commented 1 year ago

Is there any way we can get answer only from the provided sources. I tried giving a proper command in the system message of chatreadretrieveread script but it still provides me with the answers and it provides citations also. Yeah it happens less now but its still happening sometimes.

GauravSingh1A commented 1 year ago

Yeah, I would also love to know how we can do that.

pamelafox commented 1 year ago

Can you share examples of where it's answering questions that aren't in the provided sources? Have you tried using GPT-4? (It is generally better at sticking to instructions).

viru2206 commented 1 year ago

Yeah sure, I have attached an image in which you can see that it provided reponse that are not from PDF and also gave citations. The citation does not have any information regarding the query. Also, the GPT knows that the answers are not from the sources. No, I haven't tried GPT-4, I have applied for GPT-4 access but hasn't got yet. Screenshot (314)

shamprasadrh commented 1 year ago

can you try setting temperature to 0 ?

viru2206 commented 1 year ago

Yeah, I already tried that.

Kesavaraja commented 1 year ago

I am also facing the same issue. I got the content and citations from Wikipedia. Which is weird for me.

sabrab commented 1 year ago

Same, any update on this?

FranciscoMMendes commented 1 year ago

I am also facing the same issue on my side. I have tried to improve the prompt but it does not change the reply and the citations provided. When I ask the chat a question about some personality or celebrity for example, the chat fetches the information from Wikipedia instead of searching the documents provided.

chriskabat63 commented 1 year ago

I added a little more to the question prompt and seem to get pretty good results. It seems to still honor the system message, but I think this helps reinforce that it should use the included sources.

Here is kind of what the prompt looks like now:

Answer the following question only using the sources listed below:

Question: 1933 superbowl winner

Sources:

Source 1: ….

Source 2: …

I basically just replaced line 179: https://gist.github.com/chriskabat63/59da4f18657ab2b2f043cedd18d1c9ed

github-actions[bot] commented 12 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

pamelafox commented 12 months ago

Are you all still seeing this? We may move to a slightly tweaked prompt that emphasizes coherence to sources more, you could try this prompt:

https://github.com/Azure-Samples/azure-search-openai-demo/pull/967/files#diff-d6c03003d61bd0a593ea552fb76573df32d24fe6ac495da5ac2f7250cc372b96

github-actions[bot] commented 9 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.