BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
14.07k stars 1.66k forks source link

[Bug]: Langsmith async integration doesn't register traces #6862

Open tomukmatthews opened 10 hours ago

tomukmatthews commented 10 hours ago

What happened?

I ran this, following the docs but it didn't register any trace.

Using langsmith's @traceable decorator does pick up the traces, but i currently can't send in metadata dynamically with litellm.

import asyncio
import os
import litellm

os.environ["LANGSMITH_API_KEY"] = "..."
os.environ["LANGSMITH_PROJECT"] = "test"  # defaults to litellm-completion
os.environ["LANGSMITH_TRACING_V2"] = "true"
# LLM API Keys
os.environ["OPENAI_API_KEY"] = "..."

# set langsmith as a callback, litellm will send the data to langsmith
litellm.success_callback = ["langsmith"]

async def main():
    # openai call
    response = await litellm.acompletion(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}],
        metadata={
            "run_name": "litellmRUN",  # langsmith run name
            "project_name": "test",  # langsmith project name
            "run_id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",  # langsmith run id
            "parent_run_id": "f8faf8c1-9778-49a4-9004-628cdb0047e5",  # langsmith run parent run id
            "trace_id": "df570c03-5a03-4cea-8df0-c162d05127ac",  # langsmith run trace id
            "session_id": "1ffd059c-17ea-40a8-8aef-70fd0307db82",  # langsmith run session id
            "tags": ["model1", "prod-2"],  # langsmith run tags
            "metadata": {"key1": "value1"},  # langsmith run metadata
            "dotted_order": "20240429T004912090000Z497f6eca-6276-4993-bfeb-53cbbbba6f08",
        },
    )
    print(response)

if __name__ == "__main__":
    asyncio.run(main())

Relevant log output

No response

Twitter / LinkedIn details

No response