search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.47k stars 4.8k forks source link

Update Embedding Type to Support Narrow Data Types #44161

Open ShivangiReja opened 5 months ago

ShivangiReja commented 5 months ago

Issue Description

Currently, vector embeddings in Azure SDKs use the ROM<float> type. However, embeddings can also be of narrower types such as int8, int16, and float16, which consume less memory. The Azure Search service supports these narrow types, enabling customers to manage larger vector datasets at a lower cost while maintaining fast search capabilities.

Proposed Enhancement

To enhance our SDKs, we should define a new type for embeddings that can support int8, int16, float16, float32, and potentially more in the future. This update will allow users to leverage the full capabilities of embedding models and vector databases, optimizing both performance and cost.

Priority and Justification

Since Azure OpenAI is not yet GA and we haven't released OpenAI, it is crucial to prioritize making embeddings a first-class feature of public OpenAI work. This will ensure that our customers can fully utilize the capabilities of embedding models and vector databases from the outset.

ShivangiReja commented 5 months ago

Here's the initial proposal: https://gist.github.com/ShivangiReja/ffdb80d614f89c0e0d375e81a3fe564f