Azure-Samples / chat-with-your-data-solution-accelerator

A Solution Accelerator for the RAG pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences. This includes most common requirements and best practices.
https://azure.microsoft.com/products/search
MIT License
656 stars 327 forks source link

fix: Delete from storage when files deleted from Admin site #970

Closed komalg1 closed 1 month ago

komalg1 commented 1 month ago

Purpose

Does this introduce a breaking change?

Test the code

What to Check

Verify that the following are valid

github-actions[bot] commented 1 month ago

Coverage

Coverage Report
FileStmtsMissCoverMissing
code
   app.py14140%5–8, 10–11, 14, 17–19, 22, 24, 26–27
   create_app.py137199%334
code/backend
   Admin.py21210%5–9, 11, 13–14, 17, 20–21, 23–24, 27, 34, 41, 44–46, 48, 50
code/backend/batch
   add_url_embeddings.py470100% 
   batch_push_results.py360100% 
   batch_start_processing.py260100% 
   function_app.py17170%1–8, 10, 12–14, 16, 19–22
   get_conversation_response.py320100% 
code/backend/batch/utilities/common
   answer.py28292%23, 51
   source_document.py60591%44, 47, 51, 55, 128
code/backend/batch/utilities/document_chunking
   __init__.py70100% 
   chunking_strategy.py15193%25
   document_chunking_base.py10280%10, 16
   fixed_size_overlap.py190100% 
   layout.py190100% 
   page.py170100% 
   paragraph.py9277%9, 15
   strategies.py15380%15–16, 18
code/backend/batch/utilities/document_loading
   __init__.py15193%16
   document_loading_base.py9188%13
   layout.py12558%9, 12–13, 16, 25
   read.py12558%9, 12–13, 16, 25
   strategies.py20575%17, 19, 22–23, 25
   web.py19194%23
   word_document.py251348%11–12, 22–24, 27, 30, 33–37, 45
code/backend/batch/utilities/helpers
   azure_blob_storage_client.py853163%19, 23–25, 33, 57–58, 61, 78–79, 81, 85, 172–175, 179, 182, 184, 192–196, 219, 223–227, 229
   azure_computer_vision_client.py530100% 
   azure_form_recognizer_helper.py817013%11, 13, 16–17, 25, 27, 44–45, 52–55, 60–68, 73–75, 77–78, 81, 84–86, 88–90, 93, 97–98, 105–109, 111–114, 117–131, 133, 135–137, 139–140, 143, 145–147
   azure_search_helper.py570100% 
   document_chunking_helper.py13192%19
   document_loading_helper.py12191%14
   env_helper.py137695%237–239, 255–257
   llm_helper.py42783%40–41, 50, 61–62, 73, 94
   orchestrator_helper.py13192%24
code/backend/batch/utilities/helpers/config
   config_helper.py1380100% 
   conversation_flow.py40100% 
   embedding_config.py12191%27
code/backend/batch/utilities/helpers/embedders
   embedder_base.py5180%7
   embedder_factory.py100100% 
   integrated_vectorization_embedder.py36391%39–41
   push_embedder.py730100% 
code/backend/batch/utilities/integrated_vectorization
   azure_search_datasource.py190100% 
   azure_search_index.py350100% 
   azure_search_indexer.py230100% 
   azure_search_skillset.py220100% 
code/backend/batch/utilities/loggers
   conversation_logger.py36294%33–34
code/backend/batch/utilities/orchestrator
   __init__.py110100% 
   lang_chain_agent.py601968%21–24, 26, 61–62, 82–85, 102–103, 106–109, 116–117
   open_ai_functions.py541179%110–112, 115, 118–119, 122, 127–129, 134
   orchestration_strategy.py50100% 
   orchestrator_base.py50198%33
   semantickernel.py540100% 
   strategies.py12283%11, 15
code/backend/batch/utilities/parser
   __init_\.py7271%7, 11
   output_parser_tool.py390100% 
   parser_base.py9277%9, 19
code/backend/batch/utilities/plugins
   chat_plugin.py150100% 
   post_answering_plugin.py80100% 
code/backend/batch/utilities/search
   azure_search_handler.py64296%25, 31
   integrated_vectorization_search_handler.py72198%35
   search.py140100% 
   search_handler_base.py461469%16–18, 21–23, 32, 36, 40, 44, 48, 52, 56, 60
code/backend/batch/utilities/tools
   answer_processing_base.py8275%8, 12
   answering_tool_base.py9277%9, 15
   content_safety_checker.py41978%19, 52–54, 57–59, 66–67
   post_prompt_tool.py170100% 
   question_answer_tool.py550100% 
   text_processing_tool.py160100% 
code/backend/pages
   01_Ingest_Data.py70700%1–10, 12–14, 16, 22, 29, 32–33, 36–39, 41–44, 48–50, 53–55, 58–65, 68–70, 72, 75–78, 81, 86–88, 90–91, 93–94, 97–98, 102–104, 109–112, 119–120, 125, 131–132
   02_Explore_Data.py26260%1–7, 9–10, 12, 18, 25, 28, 36, 38–39, 41–44, 46–49, 52–53
   03_Delete_Data.py38380%1–8, 10–12, 14, 20, 27, 30, 38, 40–41, 43–46, 48, 50–51, 55, 59–63, 65–66, 69–70, 75–77
   04_Configuration.py1351350%1–9, 11–12, 14, 21, 28, 30, 35–44, 47–48, 51–62, 64–65, 67–69, 73–74, 86–90, 93–94, 98–100, 103–104, 107–108, 111–112, 135, 137–138, 140–144, 146–149, 152–156, 163–164, 174–176, 178, 198–199, 201, 203, 209, 217, 225, 232–233, 240, 242–243, 247, 255, 261, 268, 286–290, 296–297, 316–317, 321, 323–324, 347, 381–382, 386–387, 390–391, 394–397, 399–400, 402–404, 406–409, 411–412
TOTAL248255977% 

Tests Skipped Failures Errors Time
275 0 :zzz: 0 :x: 0 :fire: 34.069s :stopwatch:
ross-p-smith commented 1 month ago

Nothing to do with the actual code - but is there something wrong with the base of what you create your PRs from? It's a new PR, but you have a lot of 3 week history on it :-) Looks strange!

komalg1 commented 1 month ago

Nothing to do with the actual code - but is there something wrong with the base of what you create your PRs from? It's a new PR, but you have a lot of 3 week history on it :-) Looks strange!

😆 I had raised a PR from this branch for this same issue 2/3 weeks back but closed it due to Liam's changes and also this branch was messed up due to some commits to my fork's main branch. My fork's main is now in sync with Samples main, so this should not happen for a new PR

github-actions[bot] commented 1 month ago

:tada: This PR is included in version 1.2.3 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket: