Support message streaming and Async

alat-rights / alana-utilities

Quick utilities for interacting with LLMs.

https://utils.alana.computer

MIT License

1 stars 2 forks source link

Support message streaming and Async #6

Closed alat-rights closed 6 months ago

alat-rights commented 6 months ago

Change gen_msg and downstream functions to support streaming

alat-rights commented 6 months ago

In line with the general alana-utils philosophy of making the standard use-case as easy as possible, I'm tempted to just have a function for printing streamed responses to console in real time.

Will have to think more about design.

alat-rights commented 6 months ago

Completed. b00ac7202c3044720e1f1c7f4af1188c4b98b2a2