groq / groq-python

The official Python Library for the Groq API
Apache License 2.0
353 stars 23 forks source link

Feature request: Streaming with tool calls #27

Closed theognis1002 closed 3 months ago

theognis1002 commented 4 months ago

please add streaming with tool calls. This will help greatly for latency in conversations with LLM. Thank you

exception=BadRequestError("Error code: 400 - {'error': {'message': 'Tool and function calls do not currently support streaming', 'type': 'invalid_request_error'}}")
ricklamers commented 4 months ago

On the roadmap!

https://twitter.com/GavinSherry/status/1776290111421829341

rodrigoGA commented 4 months ago

@ricklamers Great. For me, this is a limitation. How can I stay informed about the addition of this feature? I hope that subscribing to the request will be enough

ricklamers commented 3 months ago

We now support it!

https://x.com/GitMaxd/status/1791577093043863876

ricklamers commented 3 months ago

PS best way to get the latest info is the Discord announcements channel: https://discord.com/channels/1207099205563457597/1207101212353368125