mindspore-lab / mindone

one for all, Optimal generator with No Exception
Apache License 2.0
328 stars 62 forks source link

feat(diffusers): StableDiffusionXLInpaintPipeline implementation #475

Closed TamirBaydasov closed 1 month ago

TamirBaydasov commented 2 months ago

What does this PR do?

Fixes # (issue)

Adds # (feature)

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

@xxx

TamirBaydasov commented 2 months ago

Demo used to check image quality:

import mindspore as ms
import requests
from PIL import Image
from io import BytesIO
from mindone.diffusers.pipelines.stable_diffusion_xl import StableDiffusionXLInpaintPipeline

pipe = StableDiffusionXLInpaintPipeline.from_pretrained(
            "stabilityai/stable-diffusion-xl-base-1.0",
            mindspore_dtype=ms.float32,
            use_safetensors=True,
            )
img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png"
mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png"
img_response = requests.get(img_url, verify=False)
mask_response = requests.get(mask_url, verify=False)

init_image = Image.open(BytesIO(img_response.content)).convert("RGB")
mask_image = Image.open(BytesIO(mask_response.content)).convert("RGB")

prompt = "A majestic tiger sitting on a bench"
ms.set_context(device_target='GPU', mode=0)
image = pipe(
            prompt=prompt, image=init_image, mask_image=mask_image, num_inference_steps=20, strength=1
            ).images[0]
TamirBaydasov commented 1 month ago

Closed due to merge with https://github.com/mindspore-lab/mindone/pull/474