Support Streaming MultiTask with `response_model`

[jxnl]

Right now, the streaming feature doesn't work with the response_model in Multitask. Instead of model.from_streaming_response() -> List[obj], we can consider a more elegant approach like the following:

class User(BaseModel):
    name: str
    age: int

users: Iterable[User] = openai.chat.completion.create(
    ...
    stream=True,
    response_model=Iterable[User]
)

for user in users:
    ...

or

users: Iterable[User] = openai.chat.completion.create(
    ...
    stream=True,
    response_model=MultiTask(User)
)

for user in users:
    ...

To handle this, we can introduce a check like:

if stream=True and response_model is Iterable, List, MultiTask, or has_attr("from_streaming_response"):

I believe there is room for improvement in this developer experience.

[jxnl]

I think Magentic does something quite nice: https://github.com/jackmpcollins/magentic/blob/4fb5899ab9691fb4849f26c11db85531cda30f90/src/magentic/chat_model/function_schema.py

from collections.abc import Iterable
from time import time

from magentic import prompt
from pydantic import BaseModel

class Superhero(BaseModel):
    name: str
    age: int
    power: str
    enemies: list[str]

@prompt("Create a Superhero team named {name}.")
def create_superhero_team(name: str) -> Iterable[Superhero]:
    ...

start_time = time()
for hero in create_superhero_team("The Food Dudes"):
    print(f"{time() - start_time:.2f}s : {hero}")

# 2.23s : name='Pizza Man' age=30 power='Can shoot pizza slices from his hands' enemies=['The Hungry Horde', 'The Junk Food Gang']
# 4.03s : name='Captain Carrot' age=35 power='Super strength and agility from eating carrots' enemies=['The Sugar Squad', 'The Greasy Gang']
# 6.05s : name='Ice Cream Girl' age=25 power='Can create ice cream out of thin air' enemies=['The Hot Sauce Squad', 'The Healthy Eaters']

That is much cleaner than instructor.

MultiUser = instructor.MultiTask(User)

completion = client.chat.completions.create(
    model="gpt-4",
    temperature=0.1,
    stream=True,
    response_model=MultiUser,
    messages=[
        {
            "role": "system",
            "content": "You are a perfect entity extraction system",
        },
        {
            "role": "user",
            "content": (
                f"Consider the data below:\n{input}"
                "Correctly segment it into entitites"
                "Make sure the JSON is correct"
            ),
        },
    ],
    max_tokens=1000,
)

for user in MultiUser.from_streaming_response(completion):
    assert isinstance(user, User)
    print(user)

>>> name="Jason" "age"=10
>>> name="John" "age"=10