search

gabrielkoo / chatgpt-faq-slack-bot

A user-trainable Knowledge Base / FAQ Slack Bot on AWS SAM based on ChatGPT and Embeddings.
https://dev.to/aws-builders/enhance-your-slack-workspace-with-a-user-trainable-chatgpt-integrated-faq-bot-2pj3
MIT License
19 stars 4 forks source link

Is response streaming supported? #10

Open sundar7D0 opened 1 year ago

sundar7D0 commented 1 year ago

Since the overall delay in returning the response from ChatGPT along with the context can be huge, does the slack-bot support response streaming where the tokens are returned to user as and when they are returned from the ChatGPT API using 'stream=True' argument?

gabrielkoo commented 1 year ago

Since the overall delay in returning the response from ChatGPT along with the context can be huge, does the slack-bot support response streaming where the tokens are returned to user as and when they are returned from the ChatGPT API using 'stream=True' argument?

Yes. You can use chat.update.

  1. Send a initial response and get the ts value of the message.
  2. Set stream=True in the openai.Completion.create call
  3. Loop through the response stream, call chat.update in each loop with the latest unfinished text response from OpenAI.

That’s it! Please feel free to create a pull request if you have done so, it will help others too!