[Issue]: Previous inpainting mask stays (and is invisible)

[Woisek]

Issue Description

A couple of times now, I noticed, that when I do some inpainting, the previous drawn mask seems to stay. Only a comple removal of the image an re-inserting solves this issue. I'm not sure how this can bae. 🙄

In the attachment you can see the mage where I inpainted and then the original and two generations. Looking at the content of the circles shows the changes where the mask was from a previously inpainting. It's maybe not so noticable when looking side by side, but if the images are flipped it's very obvious.

Version Platform Description

Win10, Firefox 106 No version displayed in the console ... ?!

[vladmandic]

i do see it when you point it out (barely noticeable otherwise), but i'm afraid this is beyond me - that code is pretty old and originated in very different repo. i'm going to mark this issue as "help needed" if someone has an idea.

[Woisek]

Yeah ... I also wasn't sure if this is something you could fix, but I had to point it out since IMHO the inpaint itself as a tool would need a big overhaul, if not a complete re-write with more advanced features. But I guess this will not happen because ... Gradio ... 😐

[vladmandic]

Yeah ... I also wasn't sure if this is something you could fix, but I had to point it out since IMHO the inpaint itself as a tool would need a big overhaul, if not a complete re-write with more advanced features. But I guess this will not happen because ... Gradio ... 😐

eventually we need to move away from gradio and have a full-fledged standalone ui, like invokeai does for example. but that is a massssssive undertaking.

[Woisek]

True, true ...

[razvan-nicolae]

it's weird because this bug appeared in recent versions of gradio I assume, it wasn't there prior to 3.23 afaik

One way to solve this definitively is to embrace one of the external inpainting tools and embed it on top of the original one. The best one to use would be https://www.reddit.com/r/StableDiffusion/comments/zv8uz5/openoutpaint_a_better_way_to_do_inpainting/

Perhaps with this fork we can have this correctly inside SD, without the pain of --api, listen, errors etc.

[tripletto]

The error can also be found in automatic1111's repository.

[Woisek]

Regarding outpainting: On Huggingface is a demo that is really nice and working -> https://huggingface.co/spaces/lnyan/stablediffusion-infinity Hoped that this would be implemented some time.

[vladmandic]

@Woisek yes, that looks quite nice, hopefully someone can take a shot at integration. if not, i'll take a look once my backlog goes down.

[Woisek]

Cool! 😀

[jonas-klesen]

I can confirm I have this same issue, and also had it in the a1111 version.

[Saelfen]

Hopefully the following questions help with diagnosing the problem: Is the image being generated with xformers or sdp with memory attention? Since they're non-deterministic they sometimes introduce small localized differences between generations of the same parameters. Also, was the inpaint method "mask only" or "whole picture"?

[illusive-c]

There's a workaround. As soon as you paint anything, the "undo" button on the top right becomes visible. Once you click it, your previous masking becomes visible. Not ideal, but what can you do.

[jonas-klesen]

Hopefully the following questions help with diagnosing the problem: Is the image being generated with xformers or sdp with memory attention? Since they're non-deterministic they sometimes introduce small localized differences between generations of the same parameters. Also, was the inpaint method "mask only" or "whole picture"?

I don't think this has anything to do with the diffusion itself. It's purely an UI issue, as far as I can tell. The mask that is shown to the user is not the same as the mask that is given to the model.

I think it happens most often when inpainting something and then sending the output to inpainting again, while fiddling around with the mask.

Let me know if you can't reproduce it. I can probably figure out a minimal example.

[anapnoe]

eventually we need to move away from gradio and have a full-fledged standalone ui, like invokeai does for example. but that is a massssssive undertaking.

I am up to it but since the API is incomplete to my knowledge and there are so many extensions that use gradio I am thinking to create a parser that serialize / deserialize the components and then use the json object to rebuild the whole interface along with some cool dnd features to reorder redesign the layout

[anapnoe]

Regarding outpainting: On Huggingface is a demo that is really nice and working -> https://huggingface.co/spaces/lnyan/stablediffusion-infinity Hoped that this would be implemented some time.

The canvas is implemented with NumPy + PyScript (the project was originally implemented with ipycanvas inside a jupyter notebook), which is relatively inefficient compared with pure frontend solutions.

[ipernjakovic]

You dont have to completely remove the image, you can just click the UNDO button on the inpaint, which will then show the invisible mask and undo it. Keep clicking until there are no more masks. Just do that whenever you have a new inpaint job, to make sure there are no hidden masks.

[AGronowski]

Just add Javascript to click the clear button whenever you click the send button. Easy fix

[moxSedai]

Any progress on this front? The only way I seem to be able to erase the mask is by restarting the server entirely. The undo method and replacing the image no longer works.

[vladmandic]

no progress. i don't want to spend too much time trying to fix existing canvas as its overall pretty bad. its on my todo to completely replace current inpainting/outpainting canvas controls as they are sub-par, but list of high-level items on my backlog is quite high.

[moxSedai]

Yeah I definitely understand that! Thanks.

[vladmandic]

can you check if the issue still exists?

[vladmandic]

any updates?

[vladmandic]

@Woisek ?

[Woisek]

To be honest, I haven't noticed this lately. But the real reason is that I mostly use Adetailer and thus no longer have the need to correct the faces manually. So I can't be really sure. I suggest to close this and re-open in the case it still exists.