search

microsoft / kernel-memory

RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.
https://microsoft.github.io/kernel-memory
MIT License
1.34k stars 252 forks source link

Added bulk embedding support for azure. #526

Closed alkampfergit closed 1 month ago

alkampfergit commented 1 month ago

Embedding in azure openai (Also standard openai) supports bulk embedding where we can reduce the number of calls requesting embedding for more than one single piece of text.

This is a really needed feature when we index lots of data (extraction from big database).

High level description (Approach, Design)

Actually this is a really simple pr where I've simple ported one class I'm using right now. I've added an IBulkTextEmbeddingGenerator that actually is implemented only by Azure openai embedding. I've created an implementation of IBulkTextEmbeddingGenerator that does not rely on Semantic Kernel (Why waiting for SK to implement new feature?) but directly uses OpenAi SDK (the resulting class is simpler).

We can close the PR as is to introduce the new interface and the new class if everything seems ok (fell free to call for modification). Once the PR is closed I'll start

  1. changing injestion pipeline to use the new bulk interface
  2. Create a plain openai implementation for bulk embedding
alkampfergit commented 1 month ago

@dluc Actually I've revieved all the PR and honestly I've simplified a lot :). First of all Kernel Memory wrapper already implemented batch feature, so I can simply use that class instead of using beta version of the library. I've changed the name of the interface and added that functionality on the standard classes that uses openai.

Once this is closed I'll proceed to use the batch feature in the ingestion pipeline.