xebia-functional / xef

Building applications with LLMs through composability, in Kotlin, Scala, ...
https://xef.ai
Apache License 2.0
178 stars 15 forks source link

Revamp OpenAI Client #697

Closed nomisRev closed 6 months ago

nomisRev commented 7 months ago

This PR revamps the OpenAI client, as discussed with @raulraja and @franciscodr.

The goal is to get rid of as much of maintenance, whilst getting as much for free as we can from the generator. This PR takes a slightly different approach, it relies more on custom templating and less on "workarounds" for the templates.

By doing so we can also support oneOf by providing a template for it, since we are now writing our own template for Ktor and multiplatform we can focus exclusively on that and provide the nicest and most idiomatic out of the experience. Which means relying on Ktor types out-of-the-box where needed, and not use any abstractions in between.

The PR also tries to solve several other problems, besides fixing the issues with generating the models (oneOf, and some edge cases).

It also gives more powerful, and fine-grained control over the HttpClient. We offer a global configuration through HttpClient, and every individual method has a configure: HttpRequestBuilder.() -> Unit = {} paramater that allows customising the global configuration but we still guarantee the correct configuration for every request (request details are not overridable).

It also generates streaming functions, for those that are explicitly listed as supported streamingOps in the KMMGenerator. These return a Flow immediately, with the return type listed by streamingOps. That way it's easier to support custom data types like AssistantEvent. Streaming uses the same setup as we currently have.

In the case of a File it immediately returns Ktor's HttpResponse, and for input it uses the existing UploadFile.

The API is designed as discussed with Raul:

val openAI = OpenAI(...)
openAI.chat.createCompletion(..)
openAI.audio.createSpeech(...)

TODO: