Open Manouchehri opened 4 months ago
This was the response for context. :)
{
"id": "chatcmpl-e5a91875-5be9-4ee1-9168-79e69dd4e4e5",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "removed",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1712955115,
"model": "gemini-1.5-pro-preview-0409",
"object": "chat.completion",
"system_fingerprint": null,
"usage": {
"completion_tokens": 650,
"prompt_tokens": 616179,
"total_tokens": 616829
}
}
+1 seeing this on embedding too
spoke to the langfuse team - they will push a fix for this by end of this week
closing since this is not litellm related
Do you know which ticket was made on the Langfuse side? :)
cc @marcklingen
@Manouchehri https://github.com/langfuse/langfuse/issues/1676
Hmm, we should just truncate the input to 1MB on LiteLLM to avoid this:
What happened?
When using Gemini 1.5 Pro with huge prompts, they aren't logged to Langfuse.
Related to https://github.com/BerriAI/litellm/issues/2055.
Relevant log output
No response
Twitter / LinkedIn details
https://www.linkedin.com/in/davidmanouchehri/