yenda
commented
1 year ago This is the code I had to use to be able to consume the stream: using aleph and manifold
(defn parse-sse [sse-string]
(let [lines (str/split sse-string #"\n\n")]
(for [line lines
:let [parts (str/split line #"\n")
data (first (filter #(str/starts-with? % "data:") parts))]]
(when data
(str/replace data "data: " "")))))
(defn handle-bytebuf [bytebuf]
(let [bytes (byte-array (.readableBytes bytebuf))]
(.getBytes bytebuf 0 bytes)
(parse-sse (String. bytes "UTF-8"))))
(defn stream-chat-completion
[messages options on-next]
(let [{:keys [model temperature max-tokens]} options]
@(d/chain
(http/post url
{:headers {"api-key" key}
:form-params (cond-> {:temperature temperature
:top_p 1
:messages messages
:stream true}
max-tokens (assoc :max_tokens max-tokens))
:content-type :json
:pool raw-stream-connection-pool
})
:body
#(stream/consume (fn [chunk]
(on-next (handle-bytebuf chunk))) %))))
wkok
I am calling create-chat-completion with the options
:stream true :on-next #(log/info :on-next %)and nothing it logs, instead I have all the "chat.completion.chunk" events dumped at once after some timeI am using azure