BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
14.4k stars 1.7k forks source link

[Bug]: Claude Sonnet 3.5 v2 on Google Vertex No Longer Works #6961

Open chibitomo2 opened 21 hours ago

chibitomo2 commented 21 hours ago

What happened?

Claude 3.5 Sonnet v2 on Vertex used to work. After upgrading to the current version v15.2.15-stable it no longer works. Just sending the word "Hi" to the model throws an error.

Xnapper-2024-11-29-01 33 45

Relevant log output

Sorry, claude35sonnet-v2.0-gcp has rejected your request. Here is the error message from claude35sonnet-v2.0-gcp: litellm.BadRequestError: VertexAIException BadRequestError - {"type":"error","error":{"type":"invalid_request_error","message":"system: text content blocks must be non-empty"}} Received Model Group=claude35sonnet-v2.0-gcp Available Model Group Fallbacks=None

Twitter / LinkedIn details

No response

krrishdholakia commented 2 hours ago

do you know what the request received by litellm is?

should be printed in logs with --detailed_debug flag

krrishdholakia commented 1 hour ago

{"type":"invalid_request_error","message":"system: text content blocks must be non-empty"}}

What's the system message being passed?

krrishdholakia commented 1 hour ago

unable to repro, a call to claude sonnet 3.5 v2 works completely fine to me w/ and w/out system blocks

Screenshot 2024-11-29 at 12 53 51 PM