[Bug]: await twice for acompletion of custom model

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

Other

12.55k stars 1.46k forks source link

What happened?

Description

When using a custom model, the acompletion function returns a coroutine that returns a coroutine so we need to await twice to get the completion result.

To Reproduce Bug

python 3.10 litellm 1.38.10

import asyncio
import litellm

async def main():
    result = await (await litellm.acompletion(
        model="custom/gpt-3.5-turbo-0125",
        base_url="https://api.openai.com/v1",
        api_key="sk-*****************************************************",
        messages=[{"role": "user", "content": "hi"}],
    ))
    print(result)

asyncio.run(main())

Relevant log output

No response

Twitter / LinkedIn details