search

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
14.08k stars 1.66k forks source link

[Bug]: jina_ai Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings' #6812

Closed chymian closed 23 hours ago

chymian commented 3 days ago

What happened?

using jina_ai via litelllm throw a 400, bad request.

litellm deployment: version 1.52.0 docker: ghcr.io/berriai/litellm-database (on coolify)

config:

  - model_name: jina_ai/*
    litellm_params:
      model: jina_ai/*
      api_key: os.environ/JINA_AI_API_KEY
  - model_name: jina_ai/jina-embeddings-v3
    litellm_params:
      model: jina_ai/jina-embeddings-v3
      max_tokens: 8192

test cmd:

  curl -L -X POST 'https://api.mydomain.tld/embeddings' -H 'Authorization: Bearer sk-xxxxxxxxxxxxx' -H 'Content-Type: application/json' -d '{"input": ["hello world"], "model": "jina_ai/jina-embeddings-v3"}'|jq

throws errors, see below. using the same cmd pointing at https://api.jina.ai/v1 works flawless.

Relevant log output

Request to litellm:                                                                                                                   
litellm.aembedding(model='jina_ai/jina-embeddings-v3', max_tokens=8192, input=['hello world'], caching=True, client=None, proxy_server_request={'url': 'http://api.mydomain.tld/embeddings', 'method': 'POST', 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1
', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.
1', 'accept-encoding': 'gzip'}, 'body': {'input': ['hello world'], 'model': 'jina_ai/jina-embeddings-v3'}}, metadata={'requester_metadata': {}, 'user_api_key_hash': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_key_alias': 'all-in', 'use
r_api_key_team_id': '49c647d1-ba02-4406-b37c-e4d607750482', 'user_api_key_user_id': 'user@email', 'user_api_key_org_id': None, 'user_api_key_team_alias': 'local-dev', 'user_api_key': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_end_user
_max_budget': None, 'litellm_api_version': '1.52.0', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team_spend': 0.02430232, 'user_api_key_spend': 0.02386622, 'user_api_key_max_budget': None, 'user_api_key_metadata': {'tags': 
None}, 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto':
 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'endpoint': 'http://api.mydomain.tld/embeddings', 'litellm_parent_otel_span': None, 'requester_ip_address': '', 'model_group': 'jina_ai/jina-embeddings-v3', 'model_grou
p_size': 1, 'deployment': 'jina_ai/jina-embeddings-v3', 'model_info': {'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'db_model': False}, 'api_base': None, 'caching_groups': None}, model_info={'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd
2b04225047d93f6e5638de', 'db_model': False}, max_retries=0)                                                                           

Initialized litellm callbacks, Async Success Callbacks: ['cache', <bound method Router.deployment_callback_on_success of <litellm.router.Router object at 0x7f73bf4caf90>>, <function _PROXY_track_cost_callback at 0x7f73c1214220>, <litellm.proxy.hooks.parallel_request_li
miter._PROXY_MaxParallelRequestsHandler object at 0x7f73c1425710>, <litellm.proxy.hooks.max_budget_limiter._PROXY_MaxBudgetLimiter object at 0x7f73c26e3d50>, <litellm.proxy.hooks.cache_control_check._PROXY_CacheControlCheck object at 0x7f73c1211f90>, <litellm._service_
logger.ServiceLogging object at 0x7f73c267fe90>]                                                                                                                                                                                                                             
ASYNC kwargs[caching]: True; litellm.cache: <litellm.caching.caching.Cache object at 0x7f73c1208a10>; kwargs.get('cache'): None                                                                                                                                              

POST Request Sent from LiteLLM:                                                                                                                                                                                                                                              
curl -X POST \                                                                                                                                                                                                                                                               
https://api.jina.ai/v1/embeddings \
-d '{'model': 'jina-embeddings-v3', 'input': ['hello world'], 'max_tokens': 8192}'

RAW RESPONSE:
Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

Inside Max Parallel Request Failure Hook
user_api_key: dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8
updated_value in failure call: {'current_requests': 0, 'current_tpm': 0, 'current_rpm': 0}

Request to litellm:
litellm.aembedding(model='jina_ai/jina-embeddings-v3', max_tokens=8192, input=['hello world'], caching=True, client=None, proxy_server_request={'url': 'http://api.mydomain.tld/embeddings', 'method': 'POST', 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1
', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.
1', 'accept-encoding': 'gzip'}, 'body': {'input': ['hello world'], 'model': 'jina_ai/jina-embeddings-v3'}}, metadata={'requester_metadata': {}, 'user_api_key_hash': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_key_alias': 'all-in', 'use
r_api_key_team_id': '49c647d1-ba02-4406-b37c-e4d607750482', 'user_api_key_user_id': 'user@email', 'user_api_key_org_id': None, 'user_api_key_team_alias': 'local-dev', 'user_api_key': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_end_user
_max_budget': None, 'litellm_api_version': '1.52.0', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team_spend': 0.02430232, 'user_api_key_spend': 0.02386622, 'user_api_key_max_budget': None, 'user_api_key_metadata': {'tags': 
None}, 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto':
 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'endpoint': 'http://api.mydomain.tld/embeddings', 'litellm_parent_otel_span': None, 'requester_ip_address': '', 'model_group': 'jina_ai/jina-embeddings-v3', 'model_grou
p_size': 1, 'deployment': 'jina_ai/jina-embeddings-v3', 'model_info': {'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'db_model': False}, 'api_base': None, 'caching_groups': None, 'previous_models': [{'exception_type': 'APIConnectionError', '
exception_string': "litellm.APIConnectionError: Openai_likeException - Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400", 'proxy_server_request': {'url':
 'http://api.mydomain.tld/embeddings', 'method': 'POST', 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld',
 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'body': {'input': ['hello world'], 'model': 'jina_ai/jina-embeddings-v3'}}, 'metadata': {'requester_metadata': {}, 'us
er_api_key_hash': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_key_alias': 'all-in', 'user_api_key_team_id': '49c647d1-ba02-4406-b37c-e4d607750482', 'user_api_key_user_id': 'user@email', 'user_api_key_org_id': None, 'user_api_key_team_a
lias': 'local-dev', 'user_api_key': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_end_user_max_budget': None, 'litellm_api_version': '1.52.0', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team
_spend': 0.02430232, 'user_api_key_spend': 0.02386622, 'user_api_key_max_budget': None, 'user_api_key_metadata': {'tags': None}, 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/jso
n', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'endpoint': 'http://api.mydomain.tld/embedding
s', 'litellm_parent_otel_span': None, 'requester_ip_address': '', 'model_group': 'jina_ai/jina-embeddings-v3', 'model_group_size': 1, 'deployment': 'jina_ai/jina-embeddings-v3', 'model_info': {'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'd
b_model': False}, 'api_base': None, 'caching_groups': None}, 'model': 'jina_ai/jina-embeddings-v3', 'input': ['hello world']}]}, model_info={'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'db_model': False}, max_retries=0)

Initialized litellm callbacks, Async Success Callbacks: ['cache', <bound method Router.deployment_callback_on_success of <litellm.router.Router object at 0x7f73bf4caf90>>, <function _PROXY_track_cost_callback at 0x7f73c1214220>, <litellm.proxy.hooks.parallel_request_li
miter._PROXY_MaxParallelRequestsHandler object at 0x7f73c1425710>, <litellm.proxy.hooks.max_budget_limiter._PROXY_MaxBudgetLimiter object at 0x7f73c26e3d50>, <litellm.proxy.hooks.cache_control_check._PROXY_CacheControlCheck object at 0x7f73c1211f90>, <litellm._service_
logger.ServiceLogging object at 0x7f73c267fe90>, <bound method SlackAlerting.response_taking_too_long_callback of <litellm.integrations.SlackAlerting.slack_alerting.SlackAlerting object at 0x7f73c13fabd0>>]
ASYNC kwargs[caching]: True; litellm.cache: <litellm.caching.caching.Cache object at 0x7f73c1208a10>; kwargs.get('cache'): None

POST Request Sent from LiteLLM:
curl -X POST \
https://api.jina.ai/v1/embeddings \
-d '{'model': 'jina-embeddings-v3', 'input': ['hello world'], 'max_tokens': 8192}'

RAW RESPONSE:
Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

Inside Max Parallel Request Failure Hook
user_api_key: dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8
updated_value in failure call: {'current_requests': 0, 'current_tpm': 0, 'current_rpm': 0}

Request to litellm:
litellm.aembedding(model='jina_ai/jina-embeddings-v3', max_tokens=8192, input=['hello world'], caching=True, client=None, proxy_server_request={'url': 'http://api.mydomain.tld/embeddings', 'method': 'POST', 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1
', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.
1', 'accept-encoding': 'gzip'}, 'body': {'input': ['hello world'], 'model': 'jina_ai/jina-embeddings-v3'}}, metadata={'requester_metadata': {}, 'user_api_key_hash': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_key_alias': 'all-in', 'use
r_api_key_team_id': '49c647d1-ba02-4406-b37c-e4d607750482', 'user_api_key_user_id': 'user@email', 'user_api_key_org_id': None, 'user_api_key_team_alias': 'local-dev', 'user_api_key': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_end_user
_max_budget': None, 'litellm_api_version': '1.52.0', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team_spend': 0.02430232, 'user_api_key_spend': 0.02386622, 'user_api_key_max_budget': None, 'user_api_key_metadata': {'tags': 
None}, 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto':
 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'endpoint': 'http://api.mydomain.tld/embeddings', 'litellm_parent_otel_span': None, 'requester_ip_address': '', 'model_group': 'jina_ai/jina-embeddings-v3', 'model_grou
p_size': 1, 'deployment': 'jina_ai/jina-embeddings-v3', 'model_info': {'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'db_model': False}, 'api_base': None, 'caching_groups': None, 'previous_models': [{'exception_type': 'APIConnectionError', '
exception_string': "litellm.APIConnectionError: Openai_likeException - Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400", 'proxy_server_request': {'url':
 'http://api.mydomain.tld/embeddings', 'method': 'POST', 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld',
 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'body': {'input': ['hello world'], 'model': 'jina_ai/jina-embeddings-v3'}}, 'metadata': {'requester_metadata': {}, 'us
er_api_key_hash': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_key_alias': 'all-in', 'user_api_key_team_id': '49c647d1-ba02-4406-b37c-e4d607750482', 'user_api_key_user_id': 'user@email', 'user_api_key_org_id': None, 'user_api_key_team_a
lias': 'local-dev', 'user_api_key': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_end_user_max_budget': None, 'litellm_api_version': '1.52.0', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team
_spend': 0.02430232, 'user_api_key_spend': 0.02386622, 'user_api_key_max_budget': None, 'user_api_key_metadata': {'tags': None}, 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/jso
n', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'endpoint': 'http://api.mydomain.tld/embedding
s', 'litellm_parent_otel_span': None, 'requester_ip_address': '', 'model_group': 'jina_ai/jina-embeddings-v3', 'model_group_size': 1, 'deployment': 'jina_ai/jina-embeddings-v3', 'model_info': {'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'd
b_model': False}, 'api_base': None, 'caching_groups': None}, 'model': 'jina_ai/jina-embeddings-v3', 'input': ['hello world']}, {'exception_type': 'APIConnectionError', 'exception_string': "litellm.APIConnectionError: Openai_likeException - Client error '400 Bad Request
' for url 'https://api.jina.ai/v1/embeddings'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400", 'proxy_server_request': {'url': 'http://api.mydomain.tld/embeddings', 'method': 'POST', 'headers': {'host': 'api.mydomain.tld', 'user-a
gent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443', 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-r
eal-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'body': {'input': ['hello world'], 'model': 'jina_ai/jina-embeddings-v3'}}, 'metadata': {'requester_metadata': {}, 'user_api_key_hash': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8', 'user_api_key_
alias': 'all-in', 'user_api_key_team_id': '49c647d1-ba02-4406-b37c-e4d607750482', 'user_api_key_user_id': 'user@email', 'user_api_key_org_id': None, 'user_api_key_team_alias': 'local-dev', 'user_api_key': 'dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf
8', 'user_api_end_user_max_budget': None, 'litellm_api_version': '1.52.0', 'global_max_parallel_requests': None, 'user_api_key_team_max_budget': None, 'user_api_key_team_spend': 0.02430232, 'user_api_key_spend': 0.02386622, 'user_api_key_max_budget': None, 'user_api_ke
y_metadata': {'tags': None}, 'headers': {'host': 'api.mydomain.tld', 'user-agent': 'curl/7.88.1', 'content-length': '65', 'accept': '*/*', 'content-type': 'application/json', 'x-forwarded-for': '172.18.0.1', 'x-forwarded-host': 'api.mydomain.tld', 'x-forwarded-port': '443'
, 'x-forwarded-proto': 'https', 'x-forwarded-server': '710f6180c9d9', 'x-real-ip': '172.18.0.1', 'accept-encoding': 'gzip'}, 'endpoint': 'http://api.mydomain.tld/embeddings', 'litellm_parent_otel_span': None, 'requester_ip_address': '', 'model_group': 'jina_ai/jina-embed
dings-v3', 'model_group_size': 1, 'deployment': 'jina_ai/jina-embeddings-v3', 'model_info': {'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'db_model': False}, 'api_base': None, 'caching_groups': None}, 'model': 'jina_ai/jina-embeddings-v3', 
'input': ['hello world']}]}, model_info={'id': '7c158a075d541a061a49ed14f4f7a026d0b8ca92bd2b04225047d93f6e5638de', 'db_model': False}, max_retries=0)

Initialized litellm callbacks, Async Success Callbacks: ['cache', <bound method Router.deployment_callback_on_success of <litellm.router.Router object at 0x7f73bf4caf90>>, <function _PROXY_track_cost_callback at 0x7f73c1214220>, <litellm.proxy.hooks.parallel_request_li
miter._PROXY_MaxParallelRequestsHandler object at 0x7f73c1425710>, <litellm.proxy.hooks.max_budget_limiter._PROXY_MaxBudgetLimiter object at 0x7f73c26e3d50>, <litellm.proxy.hooks.cache_control_check._PROXY_CacheControlCheck object at 0x7f73c1211f90>, <litellm._service_
logger.ServiceLogging object at 0x7f73c267fe90>, <bound method SlackAlerting.response_taking_too_long_callback of <litellm.integrations.SlackAlerting.slack_alerting.SlackAlerting object at 0x7f73c13fabd0>>]
ASYNC kwargs[caching]: True; litellm.cache: <litellm.caching.caching.Cache object at 0x7f73c1208a10>; kwargs.get('cache'): None

POST Request Sent from LiteLLM:
curl -X POST \
https://api.jina.ai/v1/embeddings \
-d '{'model': 'jina-embeddings-v3', 'input': ['hello world'], 'max_tokens': 8192}'

RAW RESPONSE:
Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

Inside Max Parallel Request Failure Hook
user_api_key: dcdf42285bae32a39bae6f25dc469f3a573c53dd4d2bc3d02c7eaa28de5efbf8
updated_value in failure call: {'current_requests': 0, 'current_tpm': 0, 'current_rpm': 0}
INFO:     127.0.0.1:47350 - "GET /health/liveliness HTTP/1.1" 200 OK
13:22:53 - LiteLLM Proxy:ERROR: proxy_server.py:3914 - litellm.proxy.proxy_server.embeddings(): Exception occured - litellm.APIConnectionError: Openai_likeException - Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400
Received Model Group=jina_ai/jina-embeddings-v3
Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/llms/openai_like/embedding/handler.py", line 86, in aembedding
    response = await self.async_client.post(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/llms/custom_httpx/http_handler.py", line 159, in post
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/llms/custom_httpx/http_handler.py", line 119, in post
    response.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/httpx/_models.py", line 763, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

During handling of the above exception, another exception occurred: 

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 3140, in aembedding
    response = await init_response  # type: ignore
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/llms/openai_like/embedding/handler.py", line 122, in aembedding
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/llms/openai_like/embedding/handler.py", line 96, in aembedding
    raise OpenAILikeError(
INFO:     172.21.0.6:48216 - "POST /embeddings HTTP/1.1" 500 Internal Server Error
litellm.llms.openai_like.common_utils.OpenAILikeError: Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

During handling of the above exception, another exception occurred: 

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 3867, in embeddings
    responses = await llm_responses
                ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2201, in aembedding
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2190, in aembedding
    response = await self.async_function_with_fallbacks(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2838, in async_function_with_fallbacks
    raise original_exception
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2674, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3010, in async_function_with_retries
    raise original_exception
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2922, in async_function_with_retries
    response = await self.make_call(original_function, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 3019, in make_call
    response = await response
               ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2270, in _aembedding
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2257, in _aembedding
    response = await response
               ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1228, in wrapper_async
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1084, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 3158, in aembedding
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2126, in exception_type
    raise e
  File "/usr/local/lib/python3.11/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2095, in exception_type
    raise APIConnectionError(
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: Openai_likeException - Client error '400 Bad Request' for url 'https://api.jina.ai/v1/embeddings'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400
Received Model Group=jina_ai/jina-embeddings-v3
Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2

Twitter / LinkedIn details

No response

krrishdholakia commented 1 day ago

@chymian i'm pretty sure your error is related to max_tokens

krrishdholakia commented 1 day ago

yup - can confirm this works without max_tokens in the litellm_params

Screenshot 2024-11-21 at 5 10 59 AM

i'll work on making the error message more descriptive

chymian commented 23 hours ago

I can confirm as well. thx @krrishdholakia

chymian commented 23 hours ago

@krrishdholakia Q: should max_tokens not be dropped by drop_params: true, which is set?