[Bug]: huggingface embeddings broken

[dhruv-anand-aintech]

What happened?

code:

litellm.embedding(
    model="huggingface/TaylorAI/bge-micro-v2",
    input=["batch_text"],
)

Relevant log output

Request to litellm:
litellm.embedding(model='huggingface/TaylorAI/bge-micro-v2', input=['batch_text'])

self.optional_params: {}
kwargs[caching]: False; litellm.cache: None
self.optional_params: {}


POST Request Sent from LiteLLM:
curl -X POST \
https://api-inference.huggingface.co/models/TaylorAI/bge-micro-v2 \
-H 'content-type: application/json' -H 'Authorization: Bearer hf_WQdylWJwESdfqC********************' \
-d '{'inputs': ['batch_text']}'


RAW RESPONSE:
<Response [400]>

RAW RESPONSE:
["Input should be a valid dictionary or instance of SentenceSimilarityInputsCheck: received `['batch_text']` in `parameters`"]

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Logging Details: logger_fn - None | callable(logger_fn) - False
Logging Details LiteLLM-Failure Call
self.failure_callback: []

{
    "name": "APIError",
    "message": "HuggingfaceException - [\"Input should be a valid dictionary or instance of SentenceSimilarityInputsCheck: received `['batch_text']` in `parameters`\"]",
    "stack": "---------------------------------------------------------------------------
HuggingfaceError                          Traceback (most recent call last)
File ~/miniforge3/lib/python3.10/site-packages/litellm/main.py:2737, in embedding(model, input, dimensions, timeout, api_base, api_version, api_key, api_type, caching, user, custom_llm_provider, litellm_call_id, litellm_logging_obj, logger_fn, **kwargs)
   2731     api_key = (
   2732         api_key
   2733         or litellm.huggingface_key
   2734         or get_secret(\"HUGGINGFACE_API_KEY\")
   2735         or litellm.api_key
   2736     )
-> 2737     response = huggingface.embedding(
   2738         model=model,
   2739         input=input,
   2740         encoding=encoding,
   2741         api_key=api_key,
   2742         api_base=api_base,
   2743         logging_obj=logging,
   2744         model_response=EmbeddingResponse(),
   2745     )
   2746 elif custom_llm_provider == \"bedrock\":

File ~/miniforge3/lib/python3.10/site-packages/litellm/llms/huggingface_restapi.py:750, in Huggingface.embedding(self, model, input, api_key, api_base, logging_obj, model_response, encoding)
    749 if \"error\" in embeddings:
--> 750     raise HuggingfaceError(status_code=500, message=embeddings[\"error\"])
    752 output_data = []

HuggingfaceError: [\"Input should be a valid dictionary or instance of SentenceSimilarityInputsCheck: received `['batch_text']` in `parameters`\"]

During handling of the above exception, another exception occurred:

APIError                                  Traceback (most recent call last)
/var/folders/96/56v2bn1x2gjd39_zw8jp9_s80000gn/T/ipykernel_58326/3538532934.py in ?()
----> 1 litellm.embedding(
      2     model=\"huggingface/TaylorAI/bge-micro-v2\",
      3     # model=\"embed-english-v3.0\",
      4     input=[\"batch_text\"],

~/miniforge3/lib/python3.10/site-packages/litellm/utils.py in ?(*args, **kwargs)
   2848                     if (
   2849                         liteDebuggerClient and liteDebuggerClient.dashboard_url != None
   2850                     ):  # make it easy to get to the debugger logs if you've initialized it
   2851                         e.message += f\"\
 Check the log in your dashboard - {liteDebuggerClient.dashboard_url}\"
-> 2852             raise e

~/miniforge3/lib/python3.10/site-packages/litellm/utils.py in ?(*args, **kwargs)
   2848                     if (
   2849                         liteDebuggerClient and liteDebuggerClient.dashboard_url != None
   2850                     ):  # make it easy to get to the debugger logs if you've initialized it
   2851                         e.message += f\"\
 Check the log in your dashboard - {liteDebuggerClient.dashboard_url}\"
-> 2852             raise e

~/miniforge3/lib/python3.10/site-packages/litellm/main.py in ?(model, input, dimensions, timeout, api_base, api_version, api_key, api_type, caching, user, custom_llm_provider, litellm_call_id, litellm_logging_obj, logger_fn, **kwargs)
   2889             api_key=api_key,
   2890             original_response=str(e),
   2891         )
   2892         ## Map to OpenAI Exception
-> 2893         raise exception_type(
   2894             model=model, original_exception=e, custom_llm_provider=custom_llm_provider
   2895         )

~/miniforge3/lib/python3.10/site-packages/litellm/utils.py in ?(model, original_exception, custom_llm_provider, completion_kwargs)
   8340         ):
   8341             threading.Thread(target=get_all_keys, args=(e.llm_provider,)).start()
   8342         # don't let an error with mapping interrupt the user from receiving an error from the llm api calls
   8343         if exception_mapping_worked:
-> 8344             raise e
   8345         else:
   8346             raise original_exception

~/miniforge3/lib/python3.10/site-packages/litellm/utils.py in ?(model, original_exception, custom_llm_provider, completion_kwargs)
   8340         ):
   8341             threading.Thread(target=get_all_keys, args=(e.llm_provider,)).start()
   8342         # don't let an error with mapping interrupt the user from receiving an error from the llm api calls
   8343         if exception_mapping_worked:
-> 8344             raise e
   8345         else:
   8346             raise original_exception

APIError: HuggingfaceException - [\"Input should be a valid dictionary or instance of SentenceSimilarityInputsCheck: received `['batch_text']` in `parameters`\"]"
}

Twitter / LinkedIn details

No response

[krrishdholakia]

RAW RESPONSE: ["Input should be a valid dictionary or instance of SentenceSimilarityInputsCheck: received ['batch_text'] in parameters"]

@dhruv-anand-aintech is this a text-embeddings-inference provider?

[dhruv-anand-aintech]

No, it should be using the hugging face free inference API. Not sure if that's the same as tei in litellm