search

deepset-ai / haystack-core-integrations

Additional packages (components, document stores and the likes) to extend the capabilities of Haystack version 2.0 and onwards
https://haystack.deepset.ai
Apache License 2.0
100 stars 96 forks source link

VertexAIGeminiChatGenerator streaming_callback implementation (or streaming response) #936

Closed sataycat closed 2 weeks ago

sataycat commented 1 month ago

Is your feature request related to a problem? Please describe. Similar to this ticket, the VertexAIGeminiChatGenerator does have streaming_callback implementation, even though the underlying GenerativeModel does support stream=True https://github.com/deepset-ai/haystack/issues/7836

The underlying model supports an iterable chat response https://github.com/googleapis/python-aiplatform/blob/main/vertexai/generative_models/_generative_models.py#L522

Describe the solution you'd like Similar to this https://github.com/deepset-ai/haystack/pull/8054, by @Amnah199 :)

TuanaCelik commented 3 weeks ago

I think these are pretty much the same but for Google AI generators @julian-risch and @Amnah199 = https://github.com/deepset-ai/haystack-core-integrations/issues/1002