softwaremill / sttp-openai

Apache License 2.0
41 stars 9 forks source link

Add support for Ox-based streaming responses #192

Closed lbialy closed 1 month ago

lbialy commented 1 month ago

It would be nice to be able to use sync OpenAI variant with streaming responses based on Ox's Source. There's no Streams[A] variant for Ox-based streams available in base sttp library yet as of 02.09.24 so I guess it would be necessary to define it first? I imagine once Streams[A] would be available a package called sttp.openai.streaming.ox would be created and would define something like:

    def createStreamedChatCompletion(
        chatBody: ChatBody
    ): StreamRequest[Either[OpenAIException, ox.Source[ChatChunkResponse]], OxStreams] =
      client
        .createChatCompletion(OxStreams, chatBody)
        .mapResponse(mapEventToResponse)