[Integrated Vectorization] Changes for Admin Configuration page

[komalg1]

Purpose

• Resolves #762
• Related to #321
• This PR adds Max page length & overlap length config values that can be modified from Admin configuration page.
• Adds a factory method for Processing the ingested file.
• Added & updated unit tests

Does this introduce a breaking change?

[x] Yes
[ ] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

• Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

• Test the code

  If 'AZURE_SEARCH_USE_INTEGRATED_VECTORIZATION' is true, user should be able to see two input values for Max page length & overlap length

  

What to Check

Verify that the following are valid

• Any updates to the config values should be saved.

[github-actions[bot]]

Coverage Report

File	Stmts	Miss	Cover	Missing
code
app.py	14	14	0%	1–4, 6–7, 9, 12–14, 16, 18, 20–21
create_app.py	148	3	97%	199, 204, 327
code/backend
Admin.py	21	21	0%	1–5, 7, 9–10, 12, 15–16, 18–19, 22, 29, 36, 39–41, 43, 45
code/backend/batch
AddURLEmbeddings.py	28	2	92%	32–33
BatchPushResults.py	25	0	100%
BatchStartProcessing.py	19	0	100%
GetConversationResponse.py	32	3	90%	63–65
function_app.py	16	16	0%	1–8, 10, 12–13, 15, 18–21
code/backend/batch/utilities/common
Answer.py	24	1	95%	39
SourceDocument.py	59	4	93%	33, 37, 41, 130
code/backend/batch/utilities/document_chunking
DocumentChunkingBase.py	10	2	80%	10, 16
FixedSizeOverlap.py	19	0	100%
Layout.py	19	0	100%
Page.py	17	0	100%
Paragraph.py	9	9	0%	1–4, 7–9, 12, 15
Strategies.py	29	5	82%	24–25, 27, 29, 46
__init__.py	7	0	100%
code/backend/batch/utilities/document_loading
DocumentLoadingBase.py	9	1	88%	13
Layout.py	12	12	0%	1–4, 7–9, 11–13, 16, 25
Read.py	12	12	0%	1–4, 7–9, 11–13, 16, 25
Strategies.py	20	8	60%	13, 15, 17, 19, 24–25, 27, 29
Web.py	19	1	94%	23
WordDocument.py	25	25	0%	1–6, 9–12, 21–24, 26–27, 29–30, 32–37, 45
__init__.py	15	1	93%	16
code/backend/batch/utilities/helpers
AzureBlobStorageClient.py	80	30	62%	22–24, 32, 52, 55–56, 61, 65, 90–91, 93, 97, 153, 156, 175, 178, 180, 188–192, 215, 219–223, 225
AzureFormRecognizerHelper.py	81	81	0%	1–6, 9–11, 13, 16–17, 25, 27, 35, 43–45, 52–55, 60–68, 70, 73–75, 77–78, 81, 84–86, 88–90, 93, 97–98, 105–109, 111–114, 117–131, 133, 135–137, 139–140, 143, 145–147
AzureSearchHelper.py	20	0	100%
DocumentChunkingHelper.py	12	1	91%	21
DocumentLoadingHelper.py	12	1	91%	14
EnvHelper.py	125	10	92%	202, 207–208, 211–213, 222, 226–228
LLMHelper.py	40	11	72%	40–41, 50, 61–62, 73, 86–87, 94, 104, 112
OrchestratorHelper.py	12	4	66%	20–22, 25
code/backend/batch/utilities/helpers/config
ConfigHelper.py	122	0	100%
EmbeddingConfig.py	12	1	91%	27
code/backend/batch/utilities/helpers/embedders
EmbedderBase.py	5	1	80%	7
EmbedderFactory.py	10	3	70%	12–13, 15
IntegratedVectorizationEmbedder.py	31	4	87%	20, 39–41
PushEmbedder.py	42	5	88%	24–25, 51–53
code/backend/batch/utilities/integrated_vectorization
AzureSearchDatasource.py	19	0	100%
AzureSearchIndex.py	35	0	100%
AzureSearchIndexer.py	20	2	90%	47–48
AzureSearchSkillset.py	22	0	100%
code/backend/batch/utilities/loggers
ConversationLogger.py	36	28	22%	8, 11–12, 15–24, 27–30, 33–42, 46
TokenLogger.py	9	2	77%	11, 15
code/backend/batch/utilities/orchestrator
LangChainAgent.py	60	19	68%	21–24, 26, 61–62, 82–85, 102–103, 106–109, 116–117
OpenAIFunctions.py	54	54	0%	1–3, 5–10, 12, 15–18, 53, 57–59, 62, 64, 70–73, 75, 78, 85–89, 93–94, 96, 102–106, 110–112, 115, 118–119, 122, 127–129, 132–134, 137, 142
OrchestratorBase.py	52	8	84%	35, 74–76, 83–85, 95
SemanticKernel.py	52	0	100%
Strategies.py	16	10	37%	11–12, 14–16, 18–20, 22, 24
__init__.py	11	0	100%
code/backend/batch/utilities/parser
OutputParserTool.py	39	0	100%
ParserBase.py	9	2	77%	9, 19
__init__.py	7	2	71%	7, 11
code/backend/batch/utilities/plugins
ChatPlugin.py	15	0	100%
PostAnsweringPlugin.py	8	0	100%
code/backend/batch/utilities/search
AzureSearchHandler.py	40	1	97%	13
IntegratedVectorizationSearchHandler.py	43	0	100%
Search.py	33	3	90%	16, 21–22
SearchHandlerBase.py	29	8	72%	11, 15, 19, 23, 27, 31, 35, 39
code/backend/batch/utilities/tools
AnswerProcessingBase.py	8	2	75%	8, 12
AnsweringToolBase.py	9	2	77%	9, 15
ContentSafetyChecker.py	41	25	39%	16, 18–19, 24, 30–32, 35–36, 42–43, 49–54, 57–59, 61, 65–67, 69
PostPromptTool.py	22	13	40%	11, 14–15, 17–18, 22, 29, 36–37, 45, 51–52, 60
QuestionAnswerTool.py	62	0	100%
TextProcessingTool.py	16	9	43%	9, 12–15, 21, 23, 30, 37
code/backend/pages
01_Ingest_Data.py	88	88	0%	1–12, 14–16, 18, 24, 31, 34–35, 38–47, 51–53, 56–59, 61, 64, 67–73, 76–79, 82–89, 92–94, 96, 99–102, 105, 110–112, 114–115, 117–118, 121–122, 126, 129–130, 136–139, 146–147, 152, 158–159
02_Explore_Data.py	26	26	0%	1–7, 9–10, 12, 18, 25, 28, 36, 38–39, 41–44, 46–49, 52–53
03_Delete_Data.py	37	37	0%	1–7, 9–11, 13, 19, 26, 29, 37, 39–41, 43–46, 48, 50–51, 55, 59–63, 65, 68–69, 71–73
04_Configuration.py	135	135	0%	1–9, 11–12, 14, 21, 28, 30, 35–44, 47–48, 51–62, 64–65, 67–69, 73–74, 86–90, 93–94, 98–100, 103–104, 107–108, 111–112, 135, 137–138, 140–144, 146–149, 152–156, 163–164, 174–176, 178, 198–199, 201, 203, 209, 217, 225, 232–233, 240, 242–243, 247, 255, 261, 268, 286–290, 296–297, 316–317, 321, 323–324, 347, 381–382, 384–385, 388–389, 392–395, 397–398, 400–402, 404–407, 409–410
TOTAL	2265	768	66%

Tests	Skipped	Failures	Errors	Time
166	0 :zzz:	0 :x:	0 :fire:	10.732s :stopwatch: