magma-labs / magma-chat

Ruby on Rails 7-based ChatGPT Bot Platform
https://magmachat.ai
Other
209 stars 26 forks source link

Stream output using cableready instead of DB #54

Closed obie closed 1 year ago

obie commented 1 year ago

The current brute-force implementation of streaming, where each chunk of output from gpt is saved to the database works but slams the database really hard and won't scale. Change the conversation job to stream to the client using cable ready and then save the entire output once it is generated fully.

obie commented 1 year ago

This is pretty critical so going to go ahead and take care of it already.

obie commented 1 year ago

done