microsoft / rag-experiment-accelerator

The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.
https://github.com/microsoft/rag-experiment-accelerator
Other
196 stars 71 forks source link

Add Support for Custom Embeddings #829

Open raduburtea opened 1 week ago

raduburtea commented 1 week ago

Add support for custom embeddings in the embedding folder.

This new class, that inherits from EmbeddingModel, would provide a generic template for calling any custom model that is deployed as an AzureML endpoint.

A new file called custom_embedding_model.py is created in which the CustomEmbeddingModel class is defined. This class contains several methods to make an HTTP request, package the chunk to be embedded in a format readable by the endpoint and then retrieve the result from the endpoint.

Another case is added in factory.py to support this new embedding type.

Module is fully tested.

The azure_model_api_key and azure_model_api_endpoint parameters are added the Environment class. Updated README.md to reflect these changes.