Different Return Types Depending on the Request Made to the /chat/completions Endpoint in the OpenAI OpenAPI

[paulhdk]

Question

I’ve been using the swift-openapi-generator package to generate code for invoking the OpenAI OpenAPI, which is defined here on GitHub.

When invoking the /chat/completions endpoint via createChatCompletion(), you will receive a CreateChatCompletionResponse unless you specified stream: true in your request, in that case, you’ll receive several CreateChatCompletionStreamResponse objects as the response is being streamed.

This made explicit in the /chat/completion definition in OpenAI’s openapi.yml:

    /chat/completions:
        post:
            operationId: createChatCompletion
            tags:
                - Chat
            summary: Creates a model response for the given chat conversation.
            requestBody:
                required: true
                content:
                    application/json:
                        schema:
                            $ref: "#/components/schemas/CreateChatCompletionRequest"
            responses:
                "200":
                    description: OK
                    content:
                        application/json:
                            schema:
                                $ref: "#/components/schemas/CreateChatCompletionResponse"
            x-oaiMeta:
                name: Create chat completion
                group: chat
                returns: |
                    Returns a [chat completion](/docs/api-reference/chat/object) object, or a streamed sequence of [chat completion chunk](/docs/api-reference/chat/streaming) objects if the request is streamed.

AFAICT, the CreateChatCompletionStreamResponse is never linked to the createChatCompletion() function in the spec, which is why, I believe, swift-openapi-generator doesn’t have a chance of generating a createChatCompletion() implementation that returns streamed responses, and which is why, I’m only seeing a createChatCompletion()in my generated client.swiftfile that returns a ChatCompletionResponse, indepenet of whetherstream was set or not.

Is this reasoning correct? Do you have any pointers, or am I missing something?

[czechboy0]

Hi @paulhdk,

yes, it seems the OpenAPI doc hints at a streaming variant, but doesn't actually define it.

You can edit the OpenAPI document (and possibly report it upstream to the maintainers to add it there too) and change the response content type from just:

content:
  application/json:
    schema:
      $ref: "#/components/schemas/CreateChatCompletionResponse"

to something like:

content:
  application/json:
    schema:
      $ref: "#/components/schemas/CreateChatCompletionResponse"
  text/event-stream: {}

This will ask Swift OpenAPI Generator to generate a second case for the streaming variant.

You can then use the built-in support for Server-sent Events in Swift OpenAPI Runtime to parse the events generated as the CreateChatCompletionStreamResponse struct.

An example of how to use streaming is here: https://github.com/apple/swift-openapi-generator/blob/main/Examples/event-streams-client-example/Sources/EventStreamsClient/EventStreamsClient.swift#L43-L52

[paulhdk]

Thanks for you quick reply, @czechboy0!

Hi @paulhdk,

yes, it seems the OpenAPI doc hints at a streaming variant, but doesn't actually define it.

You can edit the OpenAPI document (and possibly report it upstream to the maintainers to add it there too) and change the response content type from just:

content:
  application/json:
    schema:
      $ref: "#/components/schemas/CreateChatCompletionResponse"

to something like:

content:
  application/json:
    schema:
      $ref: "#/components/schemas/CreateChatCompletionResponse"
  text/event-stream: {}

The CreateChatCompletionResponse struct is defined in the spec, so could I add it in here somehow? Maybe instead of text/event-stream: {} do:

text/event-stream:
    schema:
        $ref: "#/components/schemas/CreateChatCompletionStreamResponse"

Not sure if this is this the correct way to do this, and not sure if it’s necessary at all.

And yes, once I have something working, I’ll open a PR in OpenAI’s OpenAPI repo.

This will ask Swift OpenAPI Generator to generate a second case for the streaming variant.

You can then use the built-in support for Server-sent Events in Swift OpenAPI Runtime to parse the events generated as the CreateChatCompletionStreamResponse struct.

Nice!

An example of how to use streaming is here: https://github.com/apple/swift-openapi-generator/blob/main/Examples/event-streams-client-example/Sources/EventStreamsClient/EventStreamsClient.swift#L43-L52

Yes, I’m aware of that example. Thanks for providing this one as well as the other examples - very helpful for devs getting started with swift-openapi-generator (like myself)!

[czechboy0]

Yes you can provide the schema for the event type, but today Swift OpenAPI Generator doesn't do anything with it. But it can be helpful for adopters reading the doc, to know which type to decode the stream of JSON payloads in the SSE as.

[paulhdk]

Yes you can provide the schema for the event type, but today Swift OpenAPI Generator doesn't do anything with it. But it can be helpful for adopters reading the doc, to know which type to decode the stream of JSON payloads in the SSE as.

Brilliant - thanks for your help!

Once I’ve succesfully tested this, I’ll close this issue and open up a PR in OpenAI’s OpenAPI repo.