mindspore-lab / mindone

one for all, Optimal generator with No Exception
https://mindspore-lab.github.io/mindone/
Apache License 2.0
353 stars 67 forks source link

feat(diffusers/pipelines): add `StableDiffusion3Img2ImgPipeline` #550

Closed townwish4git closed 2 weeks ago

townwish4git commented 3 months ago

What does this PR do?

Add StableDiffusion3Img2ImgPipeline

Give it a quick try as:

import mindspore

from mindone.diffusers import StableDiffusion3Img2ImgPipeline
from mindone.diffusers.utils import load_image

model_id_or_path = "stabilityai/stable-diffusion-3-medium-diffusers"
pipe = StableDiffusion3Img2ImgPipeline.from_pretrained(model_id_or_path, mindspore_dtype=mindspore.float16)

url = "https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg"
init_image = load_image(url).resize((512, 512))

prompt = "cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k"

images = pipe(prompt=prompt, image=init_image, strength=0.95, guidance_scale=7.5)[0][0]

then you will get a pic like: sd3_i2i_0

idk what's the connection between init_image and output images, but the result does align with 🧨diffusers if random factors are controlled manully.

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

@xxx