Open walsha2 opened 10 months ago
I wasn't able to generate the necessary code to consume some of the multipart endpoints from OpenAI
@davidmigloz any more details you can provide as to what happened? I can try to recreate myself as well.
Sorry for the late reply!
Let's take the Create image edit as a case study:
openapi: 3.0.0
info:
title: OpenAI API
version: "2.0.0"
servers:
- url: https://api.openai.com/v1
paths:
/images/edits:
post:
operationId: createImageEdit
tags:
- Images
summary: Creates an edited or extended image given an original image and a prompt.
requestBody:
required: true
content:
multipart/form-data:
schema:
$ref: "#/components/schemas/CreateImageEditRequest"
responses:
"200":
description: OK
content:
application/json:
schema:
$ref: "#/components/schemas/ImagesResponse"
components:
schemas:
CreateImageEditRequest:
type: object
properties:
image:
description: The image to edit. Must be a valid PNG file, less than 4MB, and square. If mask is not provided, image must have transparency, which will be used as the mask.
type: string
format: binary
prompt:
description: A text description of the desired image(s). The maximum length is 1000 characters.
type: string
example: "A cute baby sea otter wearing a beret"
mask:
description: An additional image whose fully transparent areas (e.g. where alpha is zero) indicate where `image` should be edited. Must be a valid PNG file, less than 4MB, and have the same dimensions as `image`.
type: string
format: binary
model:
anyOf:
- type: string
- type: string
enum: ["dall-e-2"]
default: "dall-e-2"
example: "dall-e-2"
nullable: true
description: The model to use for image generation. Only `dall-e-2` is supported at this time.
n:
type: integer
minimum: 1
maximum: 10
default: 1
example: 1
nullable: true
description: The number of images to generate. Must be between 1 and 10.
size:
type: string
enum: ["256x256", "512x512", "1024x1024"]
default: "1024x1024"
example: "1024x1024"
nullable: true
description: The size of the generated images. Must be one of `256x256`, `512x512`, or `1024x1024`.
response_format:
type: string
enum: ["url", "b64_json"]
default: "url"
example: "url"
nullable: true
description: The format in which the generated images are returned. Must be one of `url` or `b64_json`.
user:
type: string
example: user-1234
description: |
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. [Learn more](/docs/guides/safety-best-practices/end-user-ids).
required:
- prompt
- image
ImagesResponse:
properties:
created:
type: integer
data:
type: array
items:
$ref: "#/components/schemas/Image"
required:
- created
- data
Image:
type: object
description: Represents the url or the content of an image generated by the OpenAI API.
properties:
b64_json:
type: string
description: The base64-encoded JSON of the generated image, if `response_format` is `b64_json`.
url:
type: string
description: The URL of the generated image, if `response_format` is `url` (default).
revised_prompt:
type: string
description: The prompt that was used to generate the image, if there was any revision to the prompt.
The generator generates the following client method:
/// Creates an edited or extended image given an original image and a prompt.
///
/// `request`: No description
///
/// `POST` `https://api.openai.com/v1/images/edits`
Future<ImagesResponse> createImageEdit({
required List<http.MultipartFile> request,
}) async {
final r = await _request(
baseUrl: 'https://api.openai.com/v1',
path: '/images/edits',
method: HttpMethod.post,
isMultipart: true,
requestType: 'multipart/form-data',
responseType: 'application/json',
body: request,
);
return ImagesResponse.fromJson(json.decode(r.body));
}
Which expects a List<http.MultipartFile>
instead of a CreateImageEditRequest
. So there's no way to provide the other fields that the endpoint expects.
I think the createImageEdit
method should still expect a CreateImageEditRequest
object, and from that object it can extract the files.
So the CreateImageEditRequest
could be like this:
@freezed
class CreateImageEditRequest with _$CreateImageEditRequest {
const CreateImageEditRequest._();
const factory CreateImageEditRequest({
/// The image to edit. Must be a valid PNG file, less than 4MB, and square. If mask is not provided, image must have transparency, which will be used as the mask.
@JsonKey(includeToJson: false)
required File image,
/// A text description of the desired image(s). The maximum length is 1000 characters.
required String prompt,
/// An additional image whose fully transparent areas (e.g. where alpha is zero) indicate where `image` should be edited. Must be a valid PNG file, less than 4MB, and have the same dimensions as `image`.
@JsonKey(includeToJson: false)
File? mask,
/// The model to use for image generation. Only `dall-e-2` is supported at this time.
@_CreateImageEditRequestModelConverter()
@JsonKey(includeIfNull: false)
@Default(
CreateImageEditRequestModel.string('dall-e-2'),
)
CreateImageEditRequestModel? model,
/// The number of images to generate. Must be between 1 and 10.
@JsonKey(includeIfNull: false) @Default(1) int? n,
/// The size of the generated images. Must be one of `256x256`, `512x512`, or `1024x1024`.
@JsonKey(
includeIfNull: false,
unknownEnumValue: JsonKey.nullForUndefinedEnumValue,
)
@Default(CreateImageEditRequestSize.v1024x1024)
CreateImageEditRequestSize? size,
/// The format in which the generated images are returned. Must be one of `url` or `b64_json`.
@JsonKey(
name: 'response_format',
includeIfNull: false,
unknownEnumValue: JsonKey.nullForUndefinedEnumValue,
)
@Default(CreateImageEditRequestResponseFormat.url)
CreateImageEditRequestResponseFormat? responseFormat,
/// A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. [Learn more](https://platform.openai.com/docs/guides/safety-best-practices/end-user-ids).
@JsonKey(includeIfNull: false) String? user,
}) = _CreateImageEditRequest;
//...
}
The only changes are:
File
instead of String
for the binary fields@JsonKey(includeToJson: false)
annotation on those fieldsAnd the createImageEdit
method could be like:
Future<ImagesResponse> createImageEdit({
required CreateImageEditRequest request,
}) async {
final r = await _request(
baseUrl: 'https://api.openai.com/v1',
path: '/images/edits',
method: HttpMethod.post,
requestType: 'multipart/form-data',
responseType: 'application/json',
body: request,
multipartFiles: [
http.MultipartFile.fromPath('image', request.image.path),
if(request.mask != null) http.MultipartFile.fromPath('mask', request.mask!.path),
],
);
return ImagesResponse.fromJson(json.decode(r.body));
}
So instead of having a bool isMultipart
param in the _request
method, it could have a List<http.MultipartFile>? multipartFiles
param. And the generator can take those files from the CreateImageEditRequest
object.
What do you think?
Multipart requests: although the client already supports multipart requests, with the current implementation I wasn't able to generate the necessary code to consume some of the multipart endpoints from OpenAI (e.g. the audio endpoints. I may spend some time in the future adding support for this.
_Originally posted by @davidmigloz in https://github.com/tazatechnology/openapi_spec/issues/32#issuecomment-1791683173_