Correct Usage of Prompt2PromptImg2ImgPipeline

Hi,

Thanks for the great work. I wonder what is the correct way of using Prompt2PromptImg2ImgPipeline in https://github.com/HaozheZhao/UltraEdit/blob/main/data_generation/sdxl_p2p_pipeline.py? I used the following code that uses the pipeline

import torch
from sdxl_p2p_pipeline import Prompt2PromptImg2ImgPipeline
from diffusers.utils import load_image
import random

pipe = Prompt2PromptImg2ImgPipeline.from_pretrained("stabilityai/sdxl-turbo",torch_dtype=torch.float16, variant="fp16").to("cuda:0")

init_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png").resize((512, 512))

num_inference_steps = random.choice([3, 5, 8])
float_guidance_scale = random.uniform(0.4, 0.8)

cross_attention_kwargs = {
            "edit_type": "refine",
            "n_self_replace": 0.3,
            "n_cross_replace": 0.3
            }

prompts = [
    "A cat",
    "A cat in a sunny day"
]

image = pipe(prompts, image=init_image, num_inference_steps=num_inference_steps, strength=0.6, guidance_scale=float_guidance_scale,cross_attention_kwargs=cross_attention_kwargs, output_type = 'pt' ).images

From https://github.com/HaozheZhao/UltraEdit/blob/main/data_generation/data_generation.py it seems like the usage is correct.. However the code above gives the following error.

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x4352 and 2816x1280)

I wonder what is the correct usage of the Prompt2PromptImg2ImgPipeline?

HaozheZhao / UltraEdit

Correct Usage of Prompt2PromptImg2ImgPipeline #12