search

m1guelpf / chatgpt-telegram

Run your own GPTChat Telegram bot, with a single command!
MIT License
3.9k stars 560 forks source link

Wrong usage of SSE #42

Closed MeteorsLiu closed 1 year ago

MeteorsLiu commented 1 year ago

Event-stream should be reused.

However, why this project didn't reuse the event-stream connection?

Every time I call SendMessage, it just creates another event-stream, which is inefficient.

This causes a serious problem "Super slow to print out the result from the OpenAI".

I tested it for many times and i believed it was caused by this wrong usage case of SSE.

m1guelpf commented 1 year ago

I may be wrong, but since SSE only allows the server to send events, we need to start a new connection every time we want to send a message from the client. This seems in line with OpenAI's implementation as well.

fiigmnt commented 1 year ago

@m1guelpf this is correct. it looks like the event stream coming from openAI is 1 event per 1 "token" (re: https://openai.com/api/pricing/).

Each new query sends a request to the conversation end point 👍

OpenAI
Pricing
OpenAI is an AI research and deployment company. Our mission is to ensure that artificial general intelligence benefits all of humanity.