Azure-Samples / chat-with-your-data-solution-accelerator

A Solution Accelerator for the RAG pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences. This includes most common requirements and best practices.
https://azure.microsoft.com/products/search
MIT License
783 stars 395 forks source link

Uploaded documents from admin app is not being indexed into AI search #1236

Closed yaowilli closed 1 week ago

yaowilli commented 1 month ago

Describe the bug

This issue is seen when documents are ingested in the Admin app, while i expect to see the documents having after ingestion in my index of the AI search service however the documents are not indexed properly

I also observe that when i ingest document there is no activities in the backend logs at all such as [add_url_embeddings], [batch_push_results] ...etc

Expected behavior

expected behavior is to be the ingested documents being properly indexed in the AI search

How does this bug make you feel?

Share a gif from giphy to tells us how you'd feel


Debugging information

Steps to reproduce

Steps to reproduce the behavior:

  1. by simple ingestion of a document into the Admin App

Screenshots

image

image

Logs

If applicable, add logs to help the engineer debug the problem.


Tasks

To be filled in by the engineer picking up the issue

Prasanjeet-Microsoft commented 3 weeks ago

@yaowilli Could you please let us know if you are still experiencing this issue? We are asking this because we were unable to replicate this issue using the steps provided, and the ingested documents were being indexed correctly in AI search. Please find below screenshots for your reference:

image Screenshot 2024-09-03 201730 Screenshot 2024-09-03 201949 image
yaowilli commented 2 weeks ago

Hello thank you for reaching out , I am currently experiencing different issues that are stopping me from re-trying to let you the current issues are showing below image image image

yaowilli commented 2 weeks ago

image

Prasanjeet-Microsoft commented 2 weeks ago

Hello @yaowilli , Could you please provide us with either your resource group name or the URL of your app service?

yaowilli commented 2 weeks ago

RG1: rg-Crypto-001 - App service: web-5da4cne5rbana.azurewebsites.net

RG2: rg-ipasWiki-001 - App service: web-ej44tydj6vdok.azurewebsites.net

Prasanjeet-Microsoft commented 2 weeks ago

Hello @yaowilli Could you please specify the tenant and subscription name under which this resource group has been deployed in Azure?

yaowilli commented 2 weeks ago

Subscription name: WilliamYao

yaowilli commented 2 weeks ago

Tenant ID: 46c98d88-e344-4ed4-8496-4ed7712e255d

Roopan-Microsoft commented 2 weeks ago

Hi @yaowilli,

As we continue to investigate the issue, could you please verify whether the keys listed below are included in your configuration? Additionally, kindly double-check that the API key values are correct.

Mainly you can validate below keys AZURE_SEARCH_KEY AZURE_SPEECH_SERVICE_KEY AZURE_BLOB_ACCOUNT_KEY AZURE_OPENAI_API_KEY

Link: Configuration Details

yaowilli commented 2 weeks ago

image

image

yaowilli commented 2 weeks ago

do you see anything wrong in my configuration?

Prasanjeet-Microsoft commented 2 weeks ago

Hello @yaowilli , Could you please change the value of USE_KEY_VAULT from false to true and then check if you can load all the pages of the admin app properly.

yaowilli commented 2 weeks ago

Hi setting the USE_KEY_VAULT to True, seems to be making difference, let me run some more testing next week and let you know thanks

yaowilli commented 2 weeks ago

would you mind sharing what was the issue ?

yaowilli commented 2 weeks ago

I just got his issue while testing the chat App image image

Prasanjeet-Microsoft commented 2 weeks ago

would you mind sharing what was the issue ?

After investigating on the issues that you mentioned above, we found that there is a problem accessing the value of keys (OPENAI_API_KEY and AZURE_BLOB_ACCOUNT_KEY). The value of these keys are retrieved from the Key Vault secret and for this process to work correctly, the USE_KEY_VAULT setting must be enabled (i.e. set to true).

Note:- If USE_KEY_VAULT is set to "true", it initializes a SecretClient object using the AZURE_KEY_VAULT_ENDPOINT environment variable and the DefaultAzureCredential.

Prasanjeet-Microsoft commented 2 weeks ago

@yaowilli Could you please provide an update on the actual issue regarding documents uploaded from the admin app not being indexed by the AI search? Additionally, could you please create a new GitHub issue for the chat app problem?

yaowilli commented 2 weeks ago

@Prasanjeet-Microsoft thank you the errors seen from the admin App are gone now thank you - I am in the process of ingesting more documents and check against the indexes count.

Prasanjeet-Microsoft commented 2 weeks ago

@yaowilli Thank you for the update. If the ingested documents are indexed correctly in the AI search, could you please go ahead and close this issue?

yaowilli commented 1 week ago

it is closed

Roopan-Microsoft commented 1 week ago

Hey thanks @yaowilli. I'm glad to hear your issue has been resolved! If you encounter any other problems, please feel free to open a new issue.

Happy to help!