Add support for chat stream

[bluepi0j]

First time write ServerSentEvent stream, not sure if it's best approach to handle this

openAI.sendChatWithStreamEnabled(with: [
    ChatMessage(role: .user, content: "write 1000 words story")
], onEventReceived: { result in
        switch result {
        case .success(let success):
            print(success.choices?.first?.delta.content ?? "")
        case .failure(let failure):
            print(failure.localizedDescription)
        }
})

// output:
Once
 upon
 a
 time
,
 in
 a
 small
 village
 nestled
 in
... 

[le0zh0u]

These codes were used in my project, and then some issues arose. Decoding errors frequently occur, and debugging revealed that the partially received JSON data from stream might be causing decoding failures.

For example, the full json is data: {"id":"chatcmpl-6yVTvD6UAXsE9uG2SmW4Tc2iuFnnT","object":"chat.completion.chunk","created":1679878715,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"role":"assistant"},"index":0,"finish_reason":null}]}. But, first time, it received data: {"id":"chatcmpl-6yVTvD6UAXs, and next time, it received E9uG2SmW4Tc2iuFnnT","object":"chat.completion.chunk","created":1679878715,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"role":"assistant"},"index":0,"finish_reason":null}]}

JsonDecode will throw error whendecoding first time.

[le0zh0u]

I added private var cacheBuffer = [String]() and func isPartialJson(string: String) in ServerSentEventsHandler in my project.

When received data is partial json, it will be appended into cacheBuffer. And cacheBuffer will clear all elements when execute processEvent() every time. When received data is not start with data: , it will combine with data incacheBuffer, and check whether it is a full json string.

Thanks for providing code about stream feature, and look forward to your codes.

[le0zh0u]

By the way, OpenAI has request rate limit, so there will be some error messages some times. So the handler should handle these error messages.

The error example:

{
    "error": {
        "message": "Rate limit reached for default-gpt-3.5-turbo in organization ........ Visit https://platform.openai.com/account/billing to add a payment method.",
        "type": "requests",
        "param": null,
        "code": null
    }
}

[le0zh0u]

And I found it need to have a method to cancel current session task. If user back to previous view and then restart a stream chat ( the first session task has not been finished), the onRecived will get both of these response streaming.

So in app, I need to let task be canceled, when user go back to previous view.

[optera15]

And I found it need to have a method to cancel current session task. If user back to previous view and then restart a stream chat ( the first session task has not been finished), the onRecived will get both of these response streaming.

So in app, I need to let task be canceled, when user go back to previous view.

Yes, there must be an API endpoint that facilitates the "Stop Generating" button in the standard OpenAI ChatGPT interface. We need to be able to call that to end a stream.

[optera15]

Be really nice if this got finished and merged. I recently graduated from the GPT-4 waitlist, and because it's much slower there are lots of timeouts now while waiting for messages to fully arrive before display. It's also a lousy experience for the user staring at an animated bubble for ages awaiting any kind of response. It's frustrating enough that I'm looking around at other projects that do support OpenAI streaming like: https://github.com/SwiftBeta/SwiftOpenAI It would be a pain in to re-implement everything I've built around OpenAISwift but for the hottest space in tech right now this sure is a sleepy project...

[tomtao2022]

can merge to main branch， bro