search

pathwaycom / llm-app

Dynamic RAG for enterprise. Ready to run with Docker,⚡in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.
https://pathway.com/developers/templates/
MIT License
3.36k stars 192 forks source link

[FEATURE] OpenAI Chat Completion response streaming support #46

Open Boburmirzo opened 11 months ago

Boburmirzo commented 11 months ago

I assume that currently, the LLM App API client wrapper for OpenAI API does not support this streaming completions feature.

It is nice to have it where we can stream ChatGPT final responses into Pathway's output connectors such as Kafka, Redpanda or Debezium.

References:

https://github.com/openai/openai-cookbook/blob/main/examples/How_to_stream_completions.ipynb https://platform.openai.com/docs/api-reference/completions/create#completions/create-stream

mihir1739 commented 8 months ago

Can I work on this? Please assign this issue to me