search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
5.94k stars 4.08k forks source link

Behavior in production different from local deployment #1621

Closed HatefulRock closed 4 months ago

HatefulRock commented 4 months ago

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Hello, I have recently set up the integrated vectorization and linked my Azure blob to a SharePoint where files can be uploaded. These files are then indexed. When I test the chatbot locally it provides correct answers and is able to correctly cite a variety of documents. However, once deployed on the Azure web app, the chatbot seems to only reference one document no matter the question or sometimes hallucinates citations. Does any one have any insight on how to tackle this problem? Also, what is the best way to debug or get logs from the app once it is deployed into production?

Thank you in advance for your help.

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?) Windows 10

azd version?

run azd version and copy paste here.

Versions

Mention any other details that might be useful

Thanks! We'll be in touch soon.

pamelafox commented 4 months ago

Hm, do you have access control and user authentication enabled? If so, it could relate to what documents it thinks you can see. You could also double check that the search service name and the search index are the same as what's being used locally. You can get more details on how to get logs from: https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/appservice.md

HatefulRock commented 4 months ago

I checked the environment variables on the azure web app portal and indeed the index name was different from the one I had in my local environment. I thought that azd deploy would update the environment variables on the Azure portal as well. Thanks for the help!

pamelafox commented 4 months ago

The "azd up" or "azd provision" should have set the index, due to this line here:

https://github.com/Azure-Samples/azure-search-openai-demo/blob/414b1a9923d677f4ad8866efc13e5dd56672abc0/infra/main.bicep#L280

However, if you only do an "azd deploy", that won't make any infrastructure changes. So perhaps you changed the index locally and only did a deploy after. I'll close, since I think this is worked out now.