No chances understanding how inpainting work...

[Renaldas111]

After 2 hours of experimenting with PS, reading and watching all tutorials, github questions etc, I still am not able to figure out what should I need to do for an inpainting to work, it is implemented here extremely unintuitive, or I just am extremely dumb :( I open an image I want to inpaint, check the inpaint in plugin, after what "Mask -- Paint White to Mask -- temporary" layer opens at 60% opacity over my picture. Should I paint white into this 60% Mask -- Paint White to Mask -- temporary layer? After this, which layer should I select? Also random pictures appearing in plugin under Image and Mask are very confusing. The button "Generate more Inpaint", "Generate Inpaint" and "Generate inpaint" randomly changes from one to another, also sometimes green, sometimes red, what adds additional confusion. And the worst nightmare is when after pressing the button you see 10 layers in PS appearing and some of them disappearing without any understandable intuition. Surely, inpainting sometimes works, but extremely randomly with random results. Explain, please, in ELI5 terms, how should we do inpainting here? I was waiting for a PS Stable Diffusion plugin, as the old one (Stable.art) has very few options, but with this kind of confusing implementation, it is a pity, I still have to work with Krita's plugin.

[adunato]

Hi @Renaldas111, sorry you're having troubles with inpainting. I agree, the experience could be smoother, but let me try explain how it works.

• As you say the "Mask -- Paint White to Mask -- temporary" layer opens up upon the selection of inpaint and that's where you should paint the mask. the opacity is set to 60% to allow the user to see through the mask however you should paint with full opacity brush set to white (I agree this is a bit confusing)
• Once you have painted the mask you should be able to just hit generate and get your inpainting going.
• Other settings to consider are: "mask blur" and "mask expansion" which will respectively blur the mask area or expand it

The points you noted are very valid and due to:

• some styling bugs in the button changes (we will work on those)
• the way the plugin works is mostly using "batchplay" photoshop actions (basically macros) to perform layer manipulation. I completely agree ideally it would be happening behind the scenes via Photoshop API, but the API is quite limited and we're forced go via the macro way. Either way, thanks for the feedback on "usability", let me know if you can get to make Inpainting work in a more consistent way otherwise happy to help.

[Renaldas111]

Ok, I tried one more time. I opened a picture I want to inpaint, selected "inpaint" in the plugin. The 60% layer was created above my picture. Two random identical pictures appeared in plugin as Image and Mask, they are not related to the inpainting I do, I am inpaining a person, and these are pictures of a window - what do these pictures mean and why they are randomly unrelated to the work I do? I inpainted the area on this layer with 100% opacity white (which still looks as 60% opaque) and pressed Generate Inpaint (in red). Photoshop opened one more Background locked layer beneath my picture and asked to select rectangular area. Which layer should I select now? I selected the picture layer, pressed Generate Inpaint, one million new layers were created, half of them immediately disappeared, some calculations were made, but all I received is several white rectangles. OK, now I selected an inpaint layer, again clicked Generate Inpaint, now there were less new layers in PS created, but again white rectangle was a result after several seconds of SD running. Image and Mask in plugin changed their images to random ones again (now mountains). Something is really wrong either with your plugin, or with my config.

Also, what is the difference between red and green Generate (More) Inpaint button, sometimes it is red, sometimes green?

[adunato]

Here's a quick demo.

You may find them annoying but in order to understand the workflow you have to ignore

• the random images (they are there only as placeholders)
• the layer masks created at runtime, like I said it's a necessary process to manipulate the layers and masks prior to sending them to the stable diffusion backend.
• the buttons colors

apologies for the grim output image in the demo, by the way 😅

[Renaldas111]

OK, but your example showed not inpainting, but img2img. When you masked the eyes of your cat with the selection on all the canvas, which layer is selected, the image, or the mask? Surely, the most terrible thing happens after pressing the button and seeing 250 layers randomly opening and closing in your PS layers, this is enough to get a heart attack :) But at least several very confusing things are clear, Image and Mask random images and button colors are there just for additional confusion, if you managed to somehow run the inpainting :)

[adunato]

I clicked the img2img checkbox by mistake and then inpainting if you look carefully. It shouldn't matter what layer is selected, the plugin takes basically a screenshot of what is visible when you press generate.

[Renaldas111]

Ok, this is what I have, can you explain me what am I doing wrong? The main confusion is - WHICH LAYER SHOULD I SELECT AND HOW LARGE SHOULD THE SELECTION ON THAT LAYER BE?

https://youtu.be/QSEQBYhoRbg

[adunato]

Thanks for posting the video. I can't notice anything wrong in your process. At the point where you hit generate the plugin should load image and masks where the placeholders are, but this is not happening on your set up so I think the result you are getting is basically a txt2img.

Do you have the latest version of Automatic1111 and plugin installed? To be honest I have never seen this behaviour before.

[Renaldas111]

Your plugin is up to date, auto1111 is updated quite recently, but I will recheck one more time. As I already wrote, there were rare occasions when inpainting happened properly, but I have not noticed what have I done or was it random or not.

Could it be associated with my system having 3 GPUs and 2 monitors, with GPU in example working for SD and GPUs for monitors being all different. I generated images from SD with RTX2080Ti, my main monitor is connected to RTX2060S and secondary one to Intel internal graphics.

[AbdullahAlfaraj]

@Renaldas111 what's your photoshop version?

[karasuuuuu]

Hi, Im having the same issue as Renaldas111, running photoshop 24.0.0

[karasuuuuu]

Your plugin is up to date, auto1111 is updated quite recently, but I will recheck one more time. As I already wrote, there were rare occasions when inpainting happened properly, but I have not noticed what have I done or was it random or not. Could it be associated with my system having 3 GPUs and 2 monitors, with GPU in example working for SD and GPUs for monitors being all different. I generated images from SD with RTX2080Ti, my main monitor is connected to RTX2060S and secondary one to Intel internal graphics.

I think I figured out the missing step, change "select smart preset" to "fill". this is not mentioned in any of the guides or steps for inpainting that I saw. Seems to work as expected now.

[Renaldas111]

Version 24.1.1. I was able to get some inpainting with an advice to set preset as "fill", but shortly it started to misbehave again. Can anybody else make a screenrecord of their inpainting procedure in slow speed?

[Renaldas111]

Ok, looks I understood one fact, I need to create a new layer manually from PS and mask on this manually created layer, not on a layer, which is created pressing "inpaint" in plugin. At least after this I can get several inpaintings. Unfortunately, after several of them "Layers" in PS are too spammed by zillion on opened new layers, and a disaster happens. Do we really need to see all these tons of layers opened and closed at lightning speed, cant this be hidden? After several generations Layers window in PS is just a mess :(

[AbdullahAlfaraj]

@Renaldas111 Several people have reported an issue with Inpainting on the new update - we have traced the issue back to a third party library called Jimp, which is a Javascript Image Processing library that we started using in version 1.2.0.

If you choose to use the same image processing method that we used in versions prior to 1.2.0, you will need to check out this link: https://github.com/AbdullahAlfaraj/Auto-Photoshop-StableDiffusion-Plugin/issues/125#issuecomment-1468552369

[Renaldas111]

As I understood some large changes were introduced into this plugin, but inpainting is still a pain in ass, there is still no intuitive way to understand how inpainting in this plugin works. Will this plugin be able to produce inpainting in the near future?

[AbdullahAlfaraj]

@Renaldas111 Create a new document make sure to change the settings in the plugin as shown in the image, and record a video of you trying to do inpainting similar to the Youtube video you shared previously: