search

inculi / Sue

A bot for iMessage (sue@robertism.com), Telegram (@ImSueBot), and Discord. Now in Elixir!
MIT License
28 stars 9 forks source link

GPT streaming #48

Open Manwholikespie opened 1 year ago

Manwholikespie commented 1 year ago

It'd be nice if we could use the stream api for GPT for at least Telegram and Discord responses. Doing this would probably require some sort of queue such that we don't go over Discord or Telegram's API limits.

Might be worth just starting with Discord for now, as I may end up replacing all the Telegram stuff later anyway.

Manwholikespie commented 1 year ago

Discord has a rate limit of 50 requests per second, so I don't think we'll need to rate limit that platform for a while. Telegram, on the other hand, seems to prefer 20 messages / minute / chat. This is phrased as being primarily for channels, but I assume it's the same for a group.