wagtail / wagtail-vector-index

Store Wagtail pages & Django models as embeddings in vector databases
https://wagtail-vector-index.readthedocs.io/en/latest/
MIT License
15 stars 10 forks source link

Add async iteration dunder methods to LiteLLMStreamingAIResponse #76

Closed tomusher closed 1 month ago

tomusher commented 1 month ago

Adds __aiter__ and __anext__ methods to LiteLLMStreamingAIResponse so that the response can properly be iterated over without using the inner stream_wrapper.

Before:

response = await chat_backend.achat(messages=messages, stream=True)

async for chunk in response.stream_wrapper:
    print(chunk)

After:

response = await chat_backend.achat(messages=messages, stream=True)

async for chunk in response:
    print(chunk)