deep-floyd / IF

Other
7.69k stars 501 forks source link

How to get output of Zero Shot Image To Image to match input image size? #54

Open SoftologyPro opened 1 year ago

SoftologyPro commented 1 year ago

How can I ensure the output image size of image to image match the input? Going on the example colab code I use this

original_image = Image.open("input.png")

text_encoder = T5EncoderModel.from_pretrained(
    "DeepFloyd/IF-I-XL-v1.0",
    subfolder="text_encoder", 
    device_map="auto", 
    load_in_8bit=True, 
    variant="8bit"
)

pipe = IFImg2ImgPipeline.from_pretrained(
    "DeepFloyd/IF-I-XL-v1.0", 
    text_encoder=text_encoder, 
    unet=None, 
    device_map="auto"
)

prompt = "anime style"

prompt_embeds, negative_embeds = pipe.encode_prompt(prompt)

pipe = IFImg2ImgPipeline.from_pretrained(
    "DeepFloyd/IF-I-XL-v1.0", 
    text_encoder=None, 
    variant="fp16", 
    torch_dtype=torch.float16, 
    device_map="auto"
)

generator = torch.Generator().manual_seed(0)

image = pipe(
    image=original_image,
    prompt_embeds=prompt_embeds,
    negative_prompt_embeds=negative_embeds, 
    output_type="pt",
    generator=generator,
).images

pil_image = pt_to_pil(image)
pil_image[0].save("output.png")

pipe = IFImg2ImgSuperResolutionPipeline.from_pretrained(
    "DeepFloyd/IF-II-L-v1.0", 
    text_encoder=None, 
    variant="fp16", 
    torch_dtype=torch.float16, 
    device_map="auto"
)

image = pipe(
    image=image,
    original_image=original_image,
    prompt_embeds=prompt_embeds,
    negative_prompt_embeds=negative_embeds, 
    generator=generator,
).images

image[0].save("output.png")

Which works, but the output size is always smaller than the input image. What am I missing?

This is the output for a 550x550 input image. output

If possible, please give full code examples too. You have a good initial code snippet on the readme for Text to Image, but then the rest of the examples are incomplete. The same sort of full code examples would be very helpful.

Gitterman69 commented 1 year ago

bump

tummypics commented 1 year ago

bump