search

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
14.07k stars 1.66k forks source link

[Bug]: Langsmith Synchronous Integration in Docs Doesn't Work #6861

Open tomukmatthews opened 10 hours ago

tomukmatthews commented 10 hours ago

What happened?

The code example in the docs: https://docs.litellm.ai/docs/observability/langsmith_integration

For integrating litellm with langsmith fails as it expects an event loop to be running here: https://github.com/BerriAI/litellm/blob/2903fd4164010645db7ea3c77ddebb2aae870cf4/litellm/integrations/argilla.py#L86

I quickly checked and creating a new event loop if one doesn't exist resolves the issue:

    loop = asyncio.get_event_loop_policy().get_event_loop()
    if not loop.is_running():
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
    loop.create_task(self.periodic_flush())

Relevant log output

File "/home/ubuntu/.cache/pypoetry/virtualenvs/unitarybot-IlzCoXi5-py3.12/lib/python3.12/site-packages/litellm/utils.py", line 1014, in wrapper
    raise e
  File "/home/ubuntu/.cache/pypoetry/virtualenvs/unitarybot-IlzCoXi5-py3.12/lib/python3.12/site-packages/litellm/utils.py", line 798, in wrapper
    logging_obj, kwargs = function_setup(
                          ^^^^^^^^^^^^^^^
  File "/home/ubuntu/.cache/pypoetry/virtualenvs/unitarybot-IlzCoXi5-py3.12/lib/python3.12/site-packages/litellm/utils.py", line 613, in function_setup
    raise e
  File "/home/ubuntu/.cache/pypoetry/virtualenvs/unitarybot-IlzCoXi5-py3.12/lib/python3.12/site-packages/litellm/utils.py", line 423, in function_setup
    callback_class = litellm.litellm_core_utils.litellm_logging._init_custom_logger_compatible_class(  # type: ignore
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.cache/pypoetry/virtualenvs/unitarybot-IlzCoXi5-py3.12/lib/python3.12/site-packages/litellm/litellm_core_utils/litellm_logging.py", line 2175, in _init_custom_logger_compatible_class
    _langsmith_logger = LangsmithLogger()
                        ^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.cache/pypoetry/virtualenvs/unitarybot-IlzCoXi5-py3.12/lib/python3.12/site-packages/litellm/integrations/langsmith.py", line 96, in __init__
    asyncio.create_task(self.periodic_flush())
  File "/home/ubuntu/.pyenv/versions/3.12.6/lib/python3.12/asyncio/tasks.py", line 417, in create_task
    loop = events.get_running_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop
sys:1: RuntimeWarning: coroutine 'CustomBatchLogger.periodic_flush' was never awaited

Twitter / LinkedIn details

No response