Better support for multipart requests

[walsha2]

Multipart requests: although the client already supports multipart requests, with the current implementation I wasn't able to generate the necessary code to consume some of the multipart endpoints from OpenAI (e.g. the audio endpoints. I may spend some time in the future adding support for this.

_Originally posted by @davidmigloz in https://github.com/tazatechnology/openapi_spec/issues/32#issuecomment-1791683173_

[walsha2]

I wasn't able to generate the necessary code to consume some of the multipart endpoints from OpenAI

@davidmigloz any more details you can provide as to what happened? I can try to recreate myself as well.

[davidmigloz]

Sorry for the late reply!

Let's take the Create image edit as a case study:

openapi: 3.0.0
info:
  title: OpenAI API
  version: "2.0.0"
servers:
  - url: https://api.openai.com/v1
paths:
  /images/edits:
    post:
      operationId: createImageEdit
      tags:
        - Images
      summary: Creates an edited or extended image given an original image and a prompt.
      requestBody:
        required: true
        content:
          multipart/form-data:
            schema:
              $ref: "#/components/schemas/CreateImageEditRequest"
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/ImagesResponse"
components:
  schemas:
    CreateImageEditRequest:
      type: object
      properties:
        image:
          description: The image to edit. Must be a valid PNG file, less than 4MB, and square. If mask is not provided, image must have transparency, which will be used as the mask.
          type: string
          format: binary
        prompt:
          description: A text description of the desired image(s). The maximum length is 1000 characters.
          type: string
          example: "A cute baby sea otter wearing a beret"
        mask:
          description: An additional image whose fully transparent areas (e.g. where alpha is zero) indicate where `image` should be edited. Must be a valid PNG file, less than 4MB, and have the same dimensions as `image`.
          type: string
          format: binary
        model:
          anyOf:
            - type: string
            - type: string
              enum: ["dall-e-2"]
          default: "dall-e-2"
          example: "dall-e-2"
          nullable: true
          description: The model to use for image generation. Only `dall-e-2` is supported at this time.
        n:
          type: integer
          minimum: 1
          maximum: 10
          default: 1
          example: 1
          nullable: true
          description: The number of images to generate. Must be between 1 and 10.
        size:
          type: string
          enum: ["256x256", "512x512", "1024x1024"]
          default: "1024x1024"
          example: "1024x1024"
          nullable: true
          description: The size of the generated images. Must be one of `256x256`, `512x512`, or `1024x1024`.
        response_format:
          type: string
          enum: ["url", "b64_json"]
          default: "url"
          example: "url"
          nullable: true
          description: The format in which the generated images are returned. Must be one of `url` or `b64_json`.
        user:
          type: string
          example: user-1234
          description: |
            A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. [Learn more](/docs/guides/safety-best-practices/end-user-ids).
      required:
        - prompt
        - image
    ImagesResponse:
      properties:
        created:
          type: integer
        data:
          type: array
          items:
            $ref: "#/components/schemas/Image"
      required:
        - created
        - data
    Image:
      type: object
      description: Represents the url or the content of an image generated by the OpenAI API.
      properties:
        b64_json:
          type: string
          description: The base64-encoded JSON of the generated image, if `response_format` is `b64_json`.
        url:
          type: string
          description: The URL of the generated image, if `response_format` is `url` (default).
        revised_prompt:
          type: string
          description: The prompt that was used to generate the image, if there was any revision to the prompt.

The generator generates the following client method:

/// Creates an edited or extended image given an original image and a prompt.
///
/// `request`: No description
///
/// `POST` `https://api.openai.com/v1/images/edits`
Future<ImagesResponse> createImageEdit({
  required List<http.MultipartFile> request,
}) async {
  final r = await _request(
    baseUrl: 'https://api.openai.com/v1',
    path: '/images/edits',
    method: HttpMethod.post,
    isMultipart: true,
    requestType: 'multipart/form-data',
    responseType: 'application/json',
    body: request,
  );
  return ImagesResponse.fromJson(json.decode(r.body));
}

Which expects a List<http.MultipartFile> instead of a CreateImageEditRequest. So there's no way to provide the other fields that the endpoint expects.

I think the createImageEdit method should still expect a CreateImageEditRequest object, and from that object it can extract the files.

So the CreateImageEditRequest could be like this:

@freezed
class CreateImageEditRequest with _$CreateImageEditRequest {
  const CreateImageEditRequest._();

  const factory CreateImageEditRequest({
    /// The image to edit. Must be a valid PNG file, less than 4MB, and square. If mask is not provided, image must have transparency, which will be used as the mask.
    @JsonKey(includeToJson: false)
    required File image,

    /// A text description of the desired image(s). The maximum length is 1000 characters.
    required String prompt,

    /// An additional image whose fully transparent areas (e.g. where alpha is zero) indicate where `image` should be edited. Must be a valid PNG file, less than 4MB, and have the same dimensions as `image`.
    @JsonKey(includeToJson: false)
    File? mask,

    /// The model to use for image generation. Only `dall-e-2` is supported at this time.
    @_CreateImageEditRequestModelConverter()
    @JsonKey(includeIfNull: false)
    @Default(
      CreateImageEditRequestModel.string('dall-e-2'),
    )
    CreateImageEditRequestModel? model,

    /// The number of images to generate. Must be between 1 and 10.
    @JsonKey(includeIfNull: false) @Default(1) int? n,

    /// The size of the generated images. Must be one of `256x256`, `512x512`, or `1024x1024`.
    @JsonKey(
      includeIfNull: false,
      unknownEnumValue: JsonKey.nullForUndefinedEnumValue,
    )
    @Default(CreateImageEditRequestSize.v1024x1024)
    CreateImageEditRequestSize? size,

    /// The format in which the generated images are returned. Must be one of `url` or `b64_json`.
    @JsonKey(
      name: 'response_format',
      includeIfNull: false,
      unknownEnumValue: JsonKey.nullForUndefinedEnumValue,
    )
    @Default(CreateImageEditRequestResponseFormat.url)
    CreateImageEditRequestResponseFormat? responseFormat,

    /// A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. [Learn more](https://platform.openai.com/docs/guides/safety-best-practices/end-user-ids).
    @JsonKey(includeIfNull: false) String? user,
  }) = _CreateImageEditRequest;

  //...
}

The only changes are:

• Using File instead of String for the binary fields
• Adding the @JsonKey(includeToJson: false) annotation on those fields

And the createImageEdit method could be like:

Future<ImagesResponse> createImageEdit({
  required CreateImageEditRequest request,
}) async {
  final r = await _request(
    baseUrl: 'https://api.openai.com/v1',
    path: '/images/edits',
    method: HttpMethod.post,
    requestType: 'multipart/form-data',
    responseType: 'application/json',
    body: request,
    multipartFiles: [
      http.MultipartFile.fromPath('image', request.image.path),
      if(request.mask != null) http.MultipartFile.fromPath('mask', request.mask!.path),
    ],
  );
  return ImagesResponse.fromJson(json.decode(r.body));
}

So instead of having a bool isMultipart param in the _request method, it could have a List<http.MultipartFile>? multipartFiles param. And the generator can take those files from the CreateImageEditRequest object.

What do you think?