instructor-ai / instructor

structured outputs for llms
https://python.useinstructor.com/
MIT License
7.83k stars 626 forks source link

Async completions type checking error #893

Open chrisgoddard opened 2 months ago

chrisgoddard commented 2 months ago

Instructor 1.3.7 Python 3.12.3

Here's my example code:

client = instructor.from_litellm(litellm.acompletion, mode=instructor.mode.Mode.PARALLEL_TOOLS)

class Person(BaseModel):
    name: str
    age: int

class Company(BaseModel):
    name: str
    location: str

r = await client.chat.completions.create( 
    model="gpt-4o-mini",
    messages=[
        {

            "role": "user",
            "content": """
            John Smith, 29, works for Acme Corporation in San Francisco.
            """

        }
    ],
    response_model=typing.Iterable[Person | Company],
)

for obj in r:
    print(obj)

Pylance gives me these errors: Screenshot 2024-07-30 at 2 35 00 PM

However this seems to happen even without the parallel tools option

Screenshot 2024-07-30 at 2 37 36 PM

Is there something different I'm supposed to do to get async type hints working correctly?

noxan commented 2 months ago

we encounter the same issue, also with the sync version: No overloads for "create" match the provided arguments.

    client = instructor.from_openai(
        openai.OpenAI(),
        mode=instructor.Mode.PARALLEL_TOOLS,
    )

    result = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"content": prompt, "role": "system"}],
        response_model=Iterable[Schema],
    )

weirdly the type hints seems to work once Iterable is removed, but that in return breaks the actual logic.

noxan commented 2 months ago

Also seems like there are a bunch of warnings for the pyright checks, see e.g. https://github.com/jxnl/instructor/actions/runs/10179361479/job/28154928602