search

withcatai / catai

Run AI ✨ assistant locally! with simple API for Node.js 🚀
https://withcatai.github.io/catai/
MIT License
457 stars 30 forks source link

API response #21

Closed mledwards closed 1 year ago

mledwards commented 1 year ago

It looks like the API streams the whole result to the server console before sending the output back as the response. Is there a way to return the results as soon as they're available?

Or if not, then to stream the results back from the API?

ido-pluto commented 1 year ago

Currently there is the API of the ui, that works with Web-Sockets, but it is more complex and not documented yet.

You can check the example used in this file: https://github.com/ido-pluto/catai/blob/main/client/catai/src/utils/chat.ts

ido-pluto commented 1 year ago

There is a new API that supports streaming, checkout the docs