mindspore-lab / mindone

one for all, Optimal generator with No Exception
Apache License 2.0
328 stars 62 forks source link

feat(diffusers): sdxl_img2img, sdxl_inpaint, sd_img2img pipelines implementation #474

Closed TamirBaydasov closed 1 month ago

TamirBaydasov commented 2 months ago

What does this PR do?

Fixes # (issue)

Adds # (feature)

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

@xxx

TamirBaydasov commented 2 months ago

Demo used to check image quality:

import mindspore as ms
import requests
from PIL import Image
from io import BytesIO
from mindone.diffusers import StableDiffusionXLImg2ImgPipeline

pipe = StableDiffusionXLImg2ImgPipeline.from_pretrained(
            "stabilityai/stable-diffusion-xl-base-1.0",
            mindspore_dtype=ms.float32,
            use_safetensors=True,
            )

prompt = "a portrait of a dog wearing a pearl earring"
url = "https://upload.wikimedia.org/wikipedia/commons/thumb/0/0f/1665_Girl_with_a_Pearl_Earring.jpg/800px-1665_Girl_with_a_Pearl_Earring.jpg"

response = requests.get(url, verify=False)
image = Image.open(BytesIO(response.content)).convert("RGB")
image.thumbnail((768, 768))
ms.set_context(device_target='CPU', mode=0)
image = pipe(
            prompt=prompt,
            image=image,
            num_inference_steps=50,
            strength=0.75,
            guidance_scale=10.5
            )[0][0]
geniuspatrick commented 2 months ago

merge #475 #476 into this pr. keeping one pr for them is better

TamirBaydasov commented 1 month ago

merge #475 #476 into this pr. keeping one pr for them is better

Updated

TamirBaydasov commented 1 month ago

Extra question: what hardware did you test on? are both fp32 and fp16 runnable?

We use Linux CPU, Linux GPU for testing. From our tests only fp32 is runnable on both platforms. That goes for the pipelines we have implemented and those that already exist in the repository.

geniuspatrick commented 1 month ago

Once the ci linting is passed, we are good.

geniuspatrick commented 1 month ago

Please run linters with pre-commit run -a

TamirBaydasov commented 1 month ago

Please run linters with pre-commit run -a

Done

geniuspatrick commented 1 month ago

Linting is failed

CaitinZhao commented 1 month ago

Modify the code according to the ci