How to customize with LoRA when use image generation task on local PC?

[istoneyou]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Android 13

MediaPipe Tasks SDK version

com.google.mediapipe:tasks-text:0.20230731

Task name (e.g. Image classification, Gesture recognition etc.)

image generation

Programming Language and version (e.g. C++, Python, Java)

Java

Describe the actual behavior

NA

Describe the expected behaviour

NA

Standalone code/steps you may have used to try to get what you need

I am tring the image generation task and I want to konw how to customize this task with Lora. 
In the code example, a task file was used. I would like to know how this task file was generated? 
Can I run the build on a local PC through Model Maker? Is there any corresponding exampl
 code for reference instead of Vertex AI.

And if I have the lora weight file such as bowls(https://storage.googleapis.com/mediapipe-
tasks/image_generator/object/pytorch_lora_weights.bin), how to convert it to a task file which
can be used in the image generation task.

Other info / Complete Logs

No response

[kuaashish]

Hi @istoneyou,

Could you kindly provide additional details? The descriptions lack clarity regarding of the image generator support you are looking on Android, Or Python, Furthermore, could you elaborate on what is meant by "building on a local PC through Model Maker"? Specifically, when referring to a local PC, are you targeting a particular platform such as Android or Python for the build process?

Thank you

[istoneyou]

Hi @istoneyou,

Could you kindly provide additional details? The descriptions lack clarity regarding of the image generator support you are looking on Android, Or Python, Furthermore, could you elaborate on what is meant by "building on a local PC through Model Maker"? Specifically, when referring to a local PC, are you targeting a particular platform such as Android or Python for the build process?

Thank you

I fine-tuning the stable diffusion with Lora by diffusers, and the safetensors file generated. I want to know how to convert the safetensors file to task file which can be used in mediapipe image generation task api. The lora weight task file used in mediapipe like this:

val options = ImageGeneratorOptions.builder()
            .setLoraWeightsFilePath(weightsPath)
            .setImageGeneratorModelDirectory(modelPath)
            .build()

[NSTiwari]

I'm looking for the same solution. I've fine-tuned the runwayml stable-diffusion-1.5 model on custom dataset and generated the LoRA weights which are of the type .safetensor.

What are the steps to convert this safetensor file into .task file so that this can be used directly in the sample MediaPipe Image Generator app? @yichunk @kuaashish

[NSTiwari]

Hi @khanhlvg, @yichunk, Can you please help with this?

How do we convert the safetensor file to .task file that is compatible with MediaPipe?