replicate / replicate-swift

Swift client for Replicate
https://replicate.com
Apache License 2.0
165 stars 35 forks source link

Posting an image to Input #68

Closed hezhez closed 11 months ago

hezhez commented 11 months ago

Hi!

I'm trying to post an image (Swift code) to input of API and it's not working, can anyone help with updated documentation or Swift code example?

This is the code i'm using:

if let data = predictionImage?.jpegData(compressionQuality: 1.0) {
      let mimeType = "image/jpeg"
      attr["image"] = "\(data.uriEncoded(mimeType: mimeType))"
}

BR, Hezi.

mattt commented 11 months ago

Hi @hezhez. What do you mean when you say it's not working? Is the prediction failing with an error? Or does the if-let statement not evaluate and set the image attribute?

hezhez commented 11 months ago

Hi @mattt :)

Thanks for the quick response.

The base64 data is sent to the API correctly, but when i go the dashboard and see the output prediction JSON i see on image parameter the base data of the image, but it's not been used for the prediction.

mattt commented 11 months ago

@hezhez Do you think it's an issue with the model, rather than the Swift library? For example, if you call SVD and give it an image input via data: URI, does it work as expected?

hezhez commented 11 months ago

@mattt actually i didn't try another model - i'll do it now. Maybe you have working Swift example to give me for a check?

mattt commented 11 months ago

@hezhez There's example code in the README for sending an image input to tencentarc/gfpgan:

let model = try await replicate.getModel("tencentarc/gfpgan")
if let latestVersion = model.latestVersion {
    let data = try! Data(contentsOf: URL(fileURLWithPath: "/path/to/image.jpg"))
    let mimeType = "image/jpeg"
    let prediction = try await replicate.createPrediction(version: latestVersion.id,
                                                       input: ["img": "\(data.uriEncoded(mimeType: mimeType))"])
    print(prediction.output)
    // ["https://replicate.com/api/models/tencentarc/gfpgan/files/85f53415-0dc7-4703-891f-1e6f912119ad/output.png"]
}
hezhez commented 11 months ago

@mattt I saw it :) It's old and not working, take a look that they are using "img" parameter which changed long time ago to "image" ...

mattt commented 11 months ago

@hezhez Sorry, I'm not sure what you mean. The latest version of gfpgan is 9283608, which defines an img input. What about the README example code isn't working?

hezhez commented 11 months ago

@mattt I was referring to sdxl model - the correct parameter for image on this model is "image" and not "img".

Eventually i managed to use the code from the example and it worked, the problem was that i didn't configure the "prompt_strength" which is very important because the default value is 0.8 and is doing destruction of the image.

Thanks for the help!

mattt commented 11 months ago

@hezhez Glad to hear that you figured out the problem. Happy to help 😄

day-dreaming-guy commented 9 months ago

Hey @mattt, love the repo. What about the new API with the Predictable protocol. For example, using UIImage is failing, should it be Data?

enum ReplicateInput: Predictable {
  static var modelID = "catio-apps/cog-photostudio-ip_adapter_face-cyberrealistic"
  static let versionID = "c176d33db6cdbea7e4d57c200149c24e7bb3dd2e2d5ace469c31b395667d97a1"

  struct Input: Codable {
      let pose_image: UIImage
      let face_image: UIImage
  }

  typealias Output = [URL]
}