Is it possible to use component refs in paths that accept multiple content types?

Question

I've run into some issues recently with the generated Swift components for paths that support multiple content types including multipart/form-data. I was wondering if I could get some advice for working with endpoints that support multiple content-types. I'll include an example below:

openapi: 3.0.3
info:
  title: Multiple content types example
  version: 1.0.0
  description: An example path which supports requests in application/json, application/x-www-form-urlencoded, and multipart/form-data
paths:
  /api/v1/example/:
    post:
      operationId: example
      tags:
        - example
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ExampleRequest'
          application/x-www-form-urlencoded:
            schema:
              $ref: '#/components/schemas/ExampleRequest'
          multipart/form-data:
            schema:
              $ref: '#/components/schemas/ExampleRequest'
      responses:
        '201':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Example'
          description: ''
components:
  schemas:
    Example:
      type: object
      properties:
        id:
          type: integer
          readOnly: true
        description:
          type: string
          maxLength: 2000
      required:
        - id
        - description
    ExampleRequest:
      type: object
      properties:
        description:
          type: string
          minLength: 1
          maxLength: 2000
      required:
        - description

In this example, the ExampleRequest component is used as a reference in all three content types. However, when used as a reference rather than inlined, the generated Swift component seems to be compatible only with multipart/form-data request content type in the generated Operations.example.Input.Body:

internal enum Components {
    internal enum Schemas {
        // ...
        @frozen internal enum ExampleRequest: Sendable, Hashable {
            internal struct descriptionPayload: Sendable, Hashable {
                internal var body: OpenAPIRuntime.HTTPBody
                internal init(body: OpenAPIRuntime.HTTPBody) {
                    self.body = body
                }
            }
            case description(OpenAPIRuntime.MultipartPart<Components.Schemas.ExampleRequest.descriptionPayload>)
            case undocumented(OpenAPIRuntime.MultipartRawPart)
        }
    }
    // ...
}

internal enum Operations {
    internal enum example {
        // ...
        internal struct Input: Sendable, Hashable {
            // ...
            @frozen internal enum Body: Sendable, Hashable {
                case json(Components.Schemas.ExampleRequest)
                case urlEncodedForm(Components.Schemas.ExampleRequest)
                case multipartForm(OpenAPIRuntime.MultipartBody<Components.Schemas.ExampleRequest>)
            }
            internal var body: Operations.example.Input.Body
            // ...
        }
    }
}

As far as I'm aware, it's not possible to use this ExampleRequest component to initialize a JSON or URL encoded request.

This doesn't appear to be an issue when the components are defined inline. Each request content type gets its own payload type.

openapi: 3.0.3
info:
  title: Multiple content types example
  version: 1.0.0
  description: An example path which supports requests in application/json, application/x-www-form-urlencoded, and multipart/form-data
paths:
  /api/v1/example/:
    post:
      operationId: example
      tags:
        - example
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                description:
                  type: string
                  minLength: 1
                  maxLength: 2000
              required:
                - description
          application/x-www-form-urlencoded:
            schema:
              type: object
              properties:
                description:
                  type: string
                  minLength: 1
                  maxLength: 2000
              required:
                - description
          multipart/form-data:
            schema:
              type: object
              properties:
                description:
                  type: string
                  minLength: 1
                  maxLength: 2000
              required:
                - description
      responses:
        '201':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Example'
          description: ''
components:
  schemas:
    Example:
      type: object
      properties:
        id:
          type: integer
          readOnly: true
        description:
          type: string
          maxLength: 2000
      required:
        - id
        - description

internal enum Operations {
    internal enum example {
        // ...
        internal struct Input: Sendable, Hashable {
            // ...
            @frozen internal enum Body: Sendable, Hashable {
                internal struct jsonPayload: Codable, Hashable, Sendable {
                    internal var description: Swift.String
                    internal init(description: Swift.String) {
                        self.description = description
                    }
                    internal enum CodingKeys: String, CodingKey {
                        case description
                    }
                }
                case json(Operations.example.Input.Body.jsonPayload)
                internal struct urlEncodedFormPayload: Codable, Hashable, Sendable {
                    internal var description: Swift.String
                    internal init(description: Swift.String) {
                        self.description = description
                    }
                    internal enum CodingKeys: String, CodingKey {
                        case description
                    }
                }
                case urlEncodedForm(Operations.example.Input.Body.urlEncodedFormPayload)
                @frozen internal enum multipartFormPayload: Sendable, Hashable {
                    internal struct descriptionPayload: Sendable, Hashable {
                        internal var body: OpenAPIRuntime.HTTPBody
                        internal init(body: OpenAPIRuntime.HTTPBody) {
                            self.body = body
                        }
                    }
                    case description(OpenAPIRuntime.MultipartPart<Operations.example.Input.Body.multipartFormPayload.descriptionPayload>)
                    case undocumented(OpenAPIRuntime.MultipartRawPart)
                }
                case multipartForm(OpenAPIRuntime.MultipartBody<Operations.example.Input.Body.multipartFormPayload>)
            }
            internal var body: Operations.example.Input.Body
            // ...
        }
        // ..
    }
}

Is there any way I can get this to work without having to inline all my components or remove content types from my paths?

EDIT: Corrected an error in the OpenAPI document with inlined components.

Hi @collin-scavone,

yes that's right - for reusable schemas, they are generated differently based on whether they're used in mutlipart vs any other content type. The reason is that for multipart, the overall schema is not generated as Codable, because each property represents a separate part with its own content type. That's in contrast to using the schema for JSON, where we generate the type as Codable. You can read up on the details in the proposal.

Before I suggest ways to fix it, can you clarify how you use the shared schema between JSON and multipart?

If we assume SharedRequest is:

components:
  schemas:
    SharedRequest:
      type: object
      properties:
        foo:
          type: string

and its JSON form thus looks like:

{"foo":"hello"}

Is it option A, that its multipart form would be a single part with the payload {"foo":"hello"}? Or option B, that each part maps to each property, so the single part called foo would contain the value "hello"?

Both can be done, but I need to know which one you're trying to achieve before I make recommendations.

Hi @czechboy0,

The default behavior for me would be Option B.

Hi @czechboy0,

I'm assuming the example you're referencing is the operationId: postExampleMultipart. I've looked at this example before. However, there is one core issue: the request schema is defined inline.

Perhaps I can provide a little bit more context as to why this is a challenge for me.

I'm using a backend framework where the default content negotiation system for all endpoints is to accept all three of: application/json, application/x-www-form-urlencoded, and multipart/form-data. In theory, it is possible for me remove support for multipart/form-data from all endpoints which do not require file uploads, but this would require some effort, and I'd like to avoid removing functionality unnecessarily if at all possible.

The second component of this challenge is that I'm using a library which introscpects my application schema and autogenerates the OpenAPI file. After some time researching solutions, I have found no configuration option in the library which allows me to opt out of reusable schemas. If a request serializer is defined for an endpoint, the library will always output the request schema as a component reference.

TL;DR: My OpenAPI file will always have endpoints that support all three of the following request types: application/json, application/x-www-form-urlencoded, and multipart/form-data, and the request schemas will be reusable schemas.

Thanks again for your help!

TL;DR: My OpenAPI file will always have endpoints that support all three of the following request types: application/json, application/x-www-form-urlencoded , and multipart/form-data, and the request schemas will be reusable schemas.

Understood, unfortunately when introducing multipart support, we chose to only support a reusable schema that's used either as a Codable struct, or a multipart payload, not both. In our research of thousands of real-world OpenAPI documents, I hadn't seen a single example of this being used in a shipping service, so it seemed like a reasonable simplification.

The second component of this challenge is that I'm using a library which introscpects my application schema and autogenerates the OpenAPI file.

Yes, unfortunately I suspect this will complicate your workflow further. We strongly recommend using spec-driven (instead of code-driven) development for the reasons explained here: https://swiftpackageindex.com/apple/swift-openapi-generator/1.2.1/documentation/swift-openapi-generator/practicing-spec-driven-api-development

And you'll find that the whole tool was designed assuming that you have full control over your OpenAPI document. This tool is not meant to second-guess the OpenAPI document, it's meant to as faithfully as possible generate Swift code for it.

I'd suggest you switch to spec-driven development and then e.g. duplicate the reusable schemas for multipart into separate ones. Alternatively, you could try to implement the support for handling both in Swift OpenAPI Generator, but it will not be trivial, as the simplifying assumption I mentioned above did allow us to remove complexity that you'd have to add back now.

I understand the benefits of spec-driven development, it is unfortunately however not an option for us at the current moment. Regardless of our methodology, I'd like you to consider that there are "valid" (it seems this is more subjective than I initially thought 😅) use-cases -- that one could feasibly conceive of during spec-driven development -- that this implementation will break. Consider this more functional example:

openapi: 3.0.3
info:
  title: Multiple content types example
  version: 1.0.0
  description: A profile creation example where a profile picture is optional
paths:
  /api/v1/profile/:
    post:
      operationId: profileCreate
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ProfileRequest'
          application/x-www-form-urlencoded:
            schema:
              $ref: '#/components/schemas/ProfileRequest'
          multipart/form-data:
            schema:
              $ref: '#/components/schemas/ProfileRequest'
      responses:
        '201':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Profile'
          description: ''
  /api/v1/profile-pic-upload/:
    post:
      operationId: profilePicUpload
      requestBody:
        required: true
        content:
          image/png:
            schema:
              type: string
              format: binary
components:
  schemas:
    Profile:
      type: object
      properties:
        id:
          type: integer
          readOnly: true
        name:
          type: string
          maxLength: 100
        bio:
          type: string
          maxLength: 2000
        profile_pic:
          type: string
          format: uri
          nullable: true
      required:
        - id
        - name
        - bio
    ProfileRequest:
      type: object
      properties:
        name:
          type: string
          maxLength: 100
        bio:
          type: string
          minLength: 1
          maxLength: 2000
        profile_pic:
          type: string
          format: binary
          nullable: true
      required:
        - name
        - bio

In this example, the three request types share the ProfileRequest schema. In most cases, a schema of a similar structure wouldn't be valid, since profile_pic is binary. However, in this case, profile_pic is optional. This allows the user the opportunity to create a profile using a JSON or URL encoded request without a profile photo and finalize their profile later with an image upload request to /api/v1/profile-pic-upload/.

If there is no plan to support this design pattern, it seems that our best option is to revert back to a pre-release version of the generator, before multipart support was added. I had previously implemented a custom middleware and multipart parser as a workaround to enable our multipart/form-data requests. This was working quite well for our purposes.

EDIT: I suppose a reference to this XKCD comic is appropriate 😅

Thanks for the details, @collin-scavone!

Yeah unfortunately I don't see us changing this easily, again because that would require quite a lot of new complexity in the generator. For anyone hitting this issue in the future, here are some other ways to address it:

Duplicate the reusable schema, one for the multipart vs the "single-part" case (JSON, URL-encoded form, etc)
Similar as 1, make one defined inline instead of reusable

One reason why I'm still struggling to grasp the use case is as you pointed out - the schema would not be valid for JSON, as it contains a binary property (doesn't matter that it's optional, the generator would fail to generate it anyway). And in turn, if there is no binary property, it might as well be JSON or application/x-www-form-urlencoded. That's not to say you're doing anything wrong, just that I'd probably pick just one for each API operation - either multipart, or "single-part" (JSON, URL-encoded form).

Hi @czechboy0,

Yes, I suppose you're right. I'll concede that this specific example isn't strong enough to warrant a design change. An optional binary component in an application/json request schema would imply that the addition of that variable should be possible, even if it isn't. There are definitely better ways to define the schema that define the desired behavior more clearly.

I've seen other generators ignore binary fields in request content types that don't support binary values, but I see I shouldn't take this sort of functionality for granted, as it clearly isn't defined behavior.

I appreciate you taking the time to give such thoughtful suggestions! It seems that a bit of left-shifting is in order on my end. I'll use this opportunity to make some more principled design decisions 😄

Thanks for your work on the project! It's saved us a ton of time in our prototyping!

apple / swift-openapi-generator

Is it possible to use component refs in paths that accept multiple content types? #549

Question