BerriAI / litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)
https://docs.litellm.ai/docs/
Other
10.05k stars 1.12k forks source link

[Feature]: `function_to_dict` supporting type unions #4249

Open jamesbraza opened 1 week ago

jamesbraza commented 1 week ago

The Feature

As of litellm==1.40.15, type unions are not supported within litellm.utils.function_to_dict. It would be great to support type unions in this utility function.

For example, the below code will currently crash due to the type union:

import litellm

def foo(arg: int | None) -> None:
    """
    Stub.

    Args:
        arg: Stub
    """

litellm.utils.function_to_dict(foo)

To show how a type union is done in JSON schema, we can refer to pydantic==2.7.4:

from pydantic import BaseModel

class Foo(BaseModel):
    bar: int | None

print(Foo.model_json_schema())

Will print (after prettifying):

{
   "properties":{
      "bar":{
         "anyOf":[
            {
               "type":"integer"
            },
            {
               "type":"null"
            }
         ],
         "title":"Bar"
      }
   },
   "required":[
      "bar"
   ],
   "title":"Foo",
   "type":"object"
}

Motivation, pitch

There are times when a function needs to support multiple types

Twitter / LinkedIn details

No response

jamesbraza commented 2 days ago

@ishaan-jaff @krrishdholakia I solved this last weekend after I really needed the signature to work. I can propagate the solution here, which involves switching from numpydoc (only numpy style) to https://github.com/rr-/docstring_parser for support of any docstring style. Are you okay with this breaking change?

ishaan-jaff commented 2 days ago

Are you okay with this breaking change?

why is it a breaking change ?

what exactly is breaking here ?

jamesbraza commented 2 days ago

Oh yeah, https://github.com/BerriAI/litellm/blob/v1.40.15/litellm/utils.py#L4376:

--- from numpydoc.docscrape import NumpyDocString
+++ from docstring_parser import parse

Basically it's changing out the package required internally to function_to_dict. Benefits are mainly that other docstring styles can be supported (e.g. Google docstrings, which I use)