BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
13.37k stars 1.57k forks source link

[Bug]: Using Claude Dev: A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again. #6007

Closed hongbo-miao closed 3 weeks ago

hongbo-miao commented 4 weeks ago

What happened?

I tried to use Claude models (provided by AWS Bedrock) through LiteLLM proxy in the VSCode plugin Claude Dev, but got error:

APl Request Failed 400 litellm.BadRequestError: BedrockException - {"message":"A conversation must alternate between user and assistant roles. Make sure the conversation alternates between user and assistant roles and try again."} Received Model Group=claude-3-opus Available Model Group Fallbacks=None

image

I created a ticket on Claude Dev side with more details at https://github.com/saoudrizwan/claude-dev/issues/433

I am not sure if related with LiteLLM, so I opened one here as well, and hopefully both sides can help resolve it together, thanks! ☺️

Relevant log output

No response

Twitter / LinkedIn details

@hongbo_miao

Manouchehri commented 3 weeks ago

I've also ran into this using Continue.dev.

krrishdholakia commented 3 weeks ago

Hey @hongbo-miao can you share what the request received by litellm looked like? it should be easy to see when running --detailed_debug

hongbo-miao commented 3 weeks ago

Hi @krrishdholakia , here is the error log: https://gist.github.com/hongbo-miao/5ab5cdbb082d529e62f6f08cce2d1f3a Also, I saw @juancarlosm may have a workaround way at https://github.com/saoudrizwan/claude-dev/issues/433#issuecomment-2388970137 which may help as well, thanks!

krrishdholakia commented 3 weeks ago

thanks - able to repro this

krrishdholakia commented 3 weeks ago

found the issue, we didn't merge the last 2 chunks correctly

Screenshot 2024-10-02 at 9 54 54 PM