search

mongodben / mongodb-oracle

The MongoDB Oracle 🧙‍♀️🔮🌱
https://mongodb-oracle.vercel.app
8 stars 3 forks source link

stream responses #8

Closed mongodben closed 1 year ago

mongodben commented 1 year ago

instead of waiting for LLM to generate response and returning to client in 1 shot, stream the response.

for example of LLM streaming, see https://platform.openai.com/docs/api-reference/completions/create#completions/create-stream

mongodben commented 1 year ago

this presumably requires both client and server work to set up both sides of the streaming.

mongodben commented 1 year ago

tutorial video using streaming that i mentioned in meeting https://www.youtube.com/watch?v=dXsZp39L2Jk