Azure-Samples / chat-with-your-data-solution-accelerator

A Solution Accelerator for the RAG pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences. This includes most common requirements and best practices.
https://azure.microsoft.com/products/search
MIT License
521 stars 274 forks source link

Remove duplicate upload_file logic from Admin Ingest page. #782

Closed superhindupur closed 2 weeks ago

superhindupur commented 2 weeks ago

Purpose

While looking at setting up tests for the admin app pages under #736, I realized that we can refactor the pages to have a bare minimum of business logic in them, with the logic coming from the backend utilities package.

In this PR, I've removed the dupliocate upload_file to blob storage functionality and used the functionality from the AzureBlobStorageHelper.py file instead. I've also added a missing unit test to cover this functionality, achieving the overall goal of testing the admin app pages better.

Required by #736

Does this introduce a breaking change?

[ ] Yes
[x ] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[x ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

Deploy the admin app and test whether the following are working:

Other Information

github-actions[bot] commented 2 weeks ago

Coverage

Coverage Report •
FileStmtsMissCoverMissing
code/backend/batch
   BatchPushResults.py31196%53
   BatchStartProcessing.py190100% 
code/backend/batch/utilities/common
   SourceDocument.py59493%33, 37, 41, 130
code/backend/batch/utilities/helpers
   AzureBlobStorageClient.py803062%22–24, 32, 52, 55–56, 61, 65, 90–91, 93, 97, 153, 156, 175, 178, 180, 188–192, 215, 219–223, 225
   ConfigHelper.py1120100% 
code/backend/pages
   01_Ingest_Data.py88880%1–12, 14–16, 18, 24, 31, 34–35, 38–47, 51–53, 56–59, 61, 64, 67–73, 76–79, 82–89, 92–94, 96, 99–102, 105, 110–112, 114–115, 117–118, 121–122, 126, 129–130, 136–139, 146–147, 152, 158–159
TOTAL213180062% 

Tests Skipped Failures Errors Time
143 0 :zzz: 0 :x: 0 :fire: 8.537s :stopwatch:
komalg1 commented 2 weeks ago

Was this tested with Integrated Vectorization enabled for URLs?