search

Azure-Samples / azure-search-openai-demo-csharp

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure Cognitive Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
MIT License
551 stars 345 forks source link

Do you have to run Azure Embedding Service per document Page? #311

Open maximuskh opened 2 months ago

maximuskh commented 2 months ago

As I was reading through the history, I noticed that somewhere down the line, the code AzureSearchEmbedService has been updated only to call Document Intelligence per pages in the document while it used to call Document Intelligence for the whole document and use a method named BlobNameFromFilePage to decide which page is the created index record relates to?

Is there any reason why this has been updated to run per page instead of the whole document? isn't this change make it more costly to embed documents?

Whare are the pros and cons of each approach?

I appreciate if you can provide me with some information

Coruscate5 commented 2 months ago

Source doc is still included in the metadata - If you choose to use Vector (you should) as a search strategy for RAG, then the embedding model generates 1536 vectors, regardless of content length. So, for both specificity and search optimization, it makes sense to split potential search results into things that would actually be responsive AND not immediately exceed the model's context window