search

hltcoe / sandle

Run a large language modeling SANDbox in your Local Environment
Other
7 stars 1 forks source link

Document streaming usage #3

Closed ccmaymay closed 2 years ago

ccmaymay commented 2 years ago

Also maybe leave an issue or comment on OpenAI's API to use a better reference for SSE.

ccmaymay commented 2 years ago

For context, I sent the following feedback to OpenAI. This isn't so much about the API as how to use the API from the browser, however. I suspect this particular feedback is more helpful for people like me, in reimplementing the API, than for our users, who would presumably just be consuming it.

In the API documentation for the "stream" parameter of the completions endpoint, "server-sent events" links to an MDN page that describes standard server-sent events and how to use them in JavaScript. However, OpenAI's usage is somewhat nonstandard as it (a) uses a POST request rather than a GET request and (b) requires sending authorization headers, neither of which are supported in the EventSource API documented on the MDN page. I can't find a good standalone reference on consuming server-sent events using a POST request, so my suggestion would be to keep the existing link, add a comment along the lines that they can be used more generally than the EventSource API suggests, and link to something like this https://stackoverflow.com/questions/23825771/how-to-pass-post-parameters-with-html-sse/54911862#54911862 that describes how one would actually implement a consumer.