Stability-AI / stability-sdk

SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)
https://platform.stability.ai/
MIT License
2.42k stars 338 forks source link

Masked pixels get altered when using SDXL-0.9 #252

Open jfdelgad opened 1 year ago

jfdelgad commented 1 year ago

When I use SDXL-0.9 for inpainting, I'm noticing that the portions of the image covered by the mask are getting slightly altered. Interestingly, this issue doesn't occur with any other models. Ideally, the pixels of the mask with a value of 0 should remain unchanged, yet they are being modified in this scenario. Does anyone have insights into what might be happening here? Could I be making an error somewhere? For reference, I'm adhering to the examples provided for masking, but the problem persists.

jfdelgad commented 1 year ago

Does anyone have any idea? I see the same issue reported in the discord group by other people but no response at all there either.

Notice that in the example provided by the official docs the same issue is present, as areas of the image that are protected by the mask get changed.

Any help would be appreciated.

pharmapsychotic commented 1 year ago

There was a masking bug with the 2 stage models (base + refiner) 0.9 and 1.0, a fix should be rolling out before long.

UXDart commented 1 year ago

the fix will also cover old inpaint models? TIA

jfdelgad commented 1 year ago

There was a masking bug with the 2 stage models (base + refiner) 0.9 and 1.0, a fix should be rolling out before long.

@UXDart Is there any update on this?

UXDart commented 1 year ago

the black lines are fixed! now, if I create an image with SDXL 1 (btw awesome work there!) and then inpaint with (old) inpaint 2 and inpaint 1... it works, is not as good as XL 1. but if I inpaint with XL 1. it paints exactly where the mask is but it is literally following the prompt in a new image where the mask is, and not using the not masked part as "context". I could probably missing sending a var maybe? if I send a guidance var as 4-5 instead of 7 is better. but try this, inpaint at the bottom of the image. the image is a city with a street. when you inpaint at the bottom of that image, it creates houses below the street, with a "new sky" I mean is not using the context right. if I use other models inpaint or even old inpaint 2 it works better. would be great if that works as XL 1 is really good. thanks!

UXDart commented 1 year ago

interesting inpaint with 0.9 works better than 1.0 / meaning generate an image with 1.0 and then inpaint with 0.9 works better. I'm using APIs btw, not local processing etc.

UXDart commented 1 year ago

it seems this was changed again? it is all broken, before the non masked part would be changed something but at least the image made sense. now it is impossible to use, the masked part has a separate image. how I can I contact anyone? support doesn't reply at all. thanks

UXDart commented 1 year ago

biggest problem is really when inpaint at the bottom, it creates a new image, it seems to work in other directions. again can I contact anyone? thanks (reposted here)

jfdelgad commented 1 year ago

There was a masking bug with the 2 stage models (base + refiner) 0.9 and 1.0, a fix should be rolling out before long.

Any updates about this?