search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
6.06k stars 4.15k forks source link

Issues with search query generation for "chat" tab with gpt model #398

Closed hanwesh closed 1 year ago

hanwesh commented 1 year ago

Please provide us with the following information:

Screenshot 2023-07-09 173140

Minimal steps to reproduce

click on the first question, not return the right answer, do it again, it works.

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 11

azd version?

azd version 0.7.0-beta.1 (commit 9ce71659f7688d0dc3dda8b84e5accedca58cf01)

Versions

0cc155b2c8ce788be7b0916f392d8f531a71f6d9

Mention any other details that might be useful

Thanks! We'll be in touch soon.

pamelafox commented 1 year ago

This is a regression in conversation quality due to the recent change from the (deprecated) davinci model to gpt35-turbo. We need to tweak the prompt and/or do some post-processing to improve the quality of the search queries generated by gpt35-turbo.

The "ask" tab still works reliably, since it doesn't rely on gpt35-turbo to generate search queries.

Another option for you is to use this other sample, which uses cognitive search built into OpenAI: https://github.com/microsoft/sample-app-aoai-chatGPT/blob/main/README_azd.md

hanwesh commented 1 year ago

This is a regression in conversation quality due to the recent change from the (deprecated) davinci model to gpt35-turbo. We need to tweak the prompt and/or do some post-processing to improve the quality of the search queries generated by gpt35-turbo.

The "ask" tab still works reliably, since it doesn't rely on gpt35-turbo to generate search queries.

Another option for you is to use this other sample, which uses cognitive search built into OpenAI: https://github.com/microsoft/sample-app-aoai-chatGPT/blob/main/README_azd.md

you are right @pamelafox I can see the ASK still working reliably. so what is next on this? will someone tweak the prompt from the team? Thanks

pamelafox commented 1 year ago

There was a recent change by @srbalakr that attempted to improve the query generation. I also think the recent change by @pablocastro may have improved it as well.

But according to recently filed #428, it is still occasionally generating queries with quotes. @akashtalole Do you have the latest version checked out?

akashtalole commented 1 year ago

@pamelafox I have noticed that issue #428 occurs intermittently, even though I checked out the latest version today.

pamelafox commented 1 year ago

Thanks for confirming. We likely need to do additional prompt engineering or possibly post-processing.

akashtalole commented 1 year ago

I tried with this prompt line added in query_prompt_template Do not return any text inside '' or "" in the search query terms.

Even in earlier code added post processing to completion.choices[0].message.content.replace("\"", "")

Need to test above fix.

hanwesh commented 1 year ago

after get the latest version as today, the chat seems back to normal now. Let me do more testing. @pamelafox @akashtalole

pamelafox commented 1 year ago

I did a recent analysis with our current prompt between davinci and gpt-35-turbo, and I'm seeing very similar search queries generated, so I think we can close this issue. When the instruct model becomes available in the future, we'll probably try that out and see if the quality is similar/better.