Support for Text Embeddings Inference

[sivakumarl]

We are trying to integrate HF all-mpnet-base-v2 text embedding inference , fortunately able to integrate text generation inference but not embedding inference , will Nemo Guardrails support embedding inferences , appreciate your help. Thanks

Regards, Siva Kumar.

[sivakumarl]

Hi All, Able to integrate the TEI with EmbeddingModel , but not sure how to register this custom embedding class in conflig.yml for models of type embeddings . Please assist, appreciate your help. Thanks

from nemoguardrails.embeddings.embedding_providers import EmbeddingModel
from typing import List
import requests

class CustomEmbedLLM(EmbeddingModel):
    def encode( self, documents: List[str]) -> List[List[float]]:
        API_URL = "https://tsta.com/api/v1/models/all-MiniLM-L6-v2/embed"
        API_TOKEN = "abc1234"
        payload = {
            "inputs": documents
        }
        headers = {"Authorization": f"Basic {API_TOKEN}"}
        res = requests.post(API_URL, json=payload, headers=headers, verify=False)
        return res.text

Regards, Siva Kumar.

[Pouyanpi]

Hi @sivakumarl, thanks for opening this issue. This functionality will be available in the next release (0.10.0).