How to avoid character escaping in strings

[paulhdk]

Question

Hi!

I'm generating code with the OpenAI OpenAPI specification and am running into the following issue: the API allows you to pass a base64 image to the chat/completions endpoint in the image_url field of a ChatCompletionRequestMessageContentPartImage object.

        ChatCompletionRequestMessageContentPartImage:
            type: object
            title: Image content part
            properties:
                type:
                    type: string
                    enum: ["image_url"]
                    description: The type of the content part.
                image_url:
                    type: object
                    properties:
                        url:
                            type: string
                            description: Either a URL of the image or the base64 encoded image data.
                            format: uri
                        detail:
                            type: string
                            description: Specifies the detail level of the image. Learn more in the [Vision guide](/docs/guides/vision/low-or-high-fidelity-image-understanding).
                            enum: ["auto", "low", "high"]
                            default: "auto"
                    required:
                        - url
            required:
                - type
                - image_url

The base64 image string will look like the following: data:image/jpeg;base64,<base64image>.

The problem is that I'm getting a string in the final JSON where slashes, which can appear as part of a base64 string, are escaped, which causes my API calls to fail. At least I believe that is the reason why they do.

I tried changing the string format to format: byte, but not luck.

Theoretically, one would just somehow have to set the corresponding flag in the JSON encoder. The problem is that even if that were possible, I wouldn't want to disable character escaping globally but only for the specific field.

Does this make sense? Am I missing somethign? Do you have any suggestions?

[czechboy0]

Hi @paulhdk,

try to pass in custom JSON encoding options to the Configuration of the client, that might help: https://swiftpackageindex.com/apple/swift-openapi-runtime/1.5.0/documentation/openapiruntime/jsonencodingoptions/withoutescapingslashes

[paulhdk]

Hi @paulhdk,

As always, thank you for your quick reply!

try to pass in custom JSON encoding options to the Configuration of the client, that might help: https://swiftpackageindex.com/apple/swift-openapi-runtime/1.5.0/documentation/openapiruntime/jsonencodingoptions/withoutescapingslashes

Ah, I wasn't aware that you could pass configuration options to the client independently of the configuration file! I thought the configuration file was the only way to configure the code generation.

Passing JSONEncodingOptions.withoutEscapingSlashes when the client is generated will set that option globally, won't it? You aren't aware of any way where I could only unescape the image_url field?

Another approach: use a ClientMiddleware to intercept outgoing requests, check if the corresponding bytes for image_url are present in the body, and then manually decode the body, unescape the characters in the image_url field and re-encode the body. This is quite hacky and probably also comes with a performance penalty.

What do you think?

[czechboy0]

Yup it turns it on for the whole client all operations, but should be safe. Escaping forward slashes in JSON is there for historical reasons, but usually isn't necessary for correctness.

Yes, the middleware approach is the most flexible, but likely also most expensive. That said, it might still be fast enough for your use case, you'll need to try it out, if setting globally still isn't preferable for you.

[simonjbeaumont]

Hm, this OpenAPI document doesn't seem to be expressing very clearly the intent of that operation.

url:
  type: string
  description: Either a URL of the image or the base64 encoded image data.
  format: uri

Having one field that is either a URL or a base64-encoded data is a quite a lift.

The OpenAPI spec does have examples of fields that explicitly use base64 encoding, e.g. in multipart:

type: string
contentMediaType: image/png
contentEncoding: base64

@paulhdk did you have issues when configuring without escaping globally for the client?

[paulhdk]

Yup it turns it on for the whole client all operations, but should be safe. Escaping forward slashes in JSON is there for historical reasons, but usually isn't necessary for correctness.

Oh. Interesting!

Yes, the middleware approach is the most flexible, but likely also most expensive. That said, it might still be fast enough for your use case, you'll need to try it out, if setting globally still isn't preferable for you.

Hm, this OpenAPI document doesn't seem to be expressing very clearly the intent of that operation.

url:
  type: string
  description: Either a URL of the image or the base64 encoded image data.
  format: uri

Having one field that is either a URL or a base64-encoded data is a quite a lift.

The OpenAPI spec does have examples of fields that explicitly use base64 encoding, e.g. in multipart:

type: string
contentMediaType: image/png
contentEncoding: base64

Oh, I didn't know a base64 content encoding existet for strings in OpenAPI. I'd only experimented with byte.

@paulhdk did you have issues when configuring without escaping globally for the client?

So far no issues! I'll close this issue for now and might re-open if I run into some problems. For our use case, we don't need the URLs anyway, so maybe we could just go for the base64 content encoding if that gets rid of the escaping too. I'll have to try that out.

Thank you both!