[Feature Request]: Mask in shape of the face (not square)

[reekes]

Is your feature request related to a problem? Please describe.

First, let me say this extension is a must-have. It's by far the one I use the most in Automatic1111.

The issue I'm raising is how faces are not squares. By creating a square mask adetailer ends up modifying parts of the image outside of the face. Adjusting the size of the mask, blur, or padding becomes a trade-off of not fixing part of the face or disturbing areas outside of the face.

I often have problems when the face and surrounding content are blended (bleeding or smearing of the image). For example, sometimes the hair around the face becomes blurred while the face is being "fixed."

The worst cases are when the angle of the face doesn't fit well within a square, such as a profile or angles where the nose extends beyond the face's outline.

PS - Square masks for hands is even worse. I've given up trying to use adetailer on hands.

Describe the solution you'd like

Generate a mask that is just the face area.

Even better, It could even be a contour (depth) map (if possible), then applying adjustments to the face as a 3D shape.

Describe alternatives you've considered

Dropping Automatic1111 and going to ComfyUI which has a node that can generate masks in the shape of the detected face.

The only other option I've got is to create my own mask by hand in img2img, then applying a fix manually. This is like getting into using Photoshop as a photographer (which I've done for decades, and it's too time consuming). Avoiding manual steps was the problem I think adetailer was designed to solve.

Additional context

No response

[Bing-su]

When I started this project, it was hard to find a dataset that segmented only faces, but now there are many community-created models available.

As an example, you can download several models from the links below and put them in the stable-diffusion-webui\models\adetailer folder. Thank you.

https://civitai.com/search/models?sortBy=models_v9&query=adetailer (NSFW)

[reekes]

Thank you Bing-su. This is one of those type of searches where you can't find what you're looking for if you don't know what to call it. I guess what I am asking about is "segmentation" not masks. OK, so I installed a couple from Anzhc. One creates a mask for the entire head and includes hair (and a few things). Mix results with that depending on the image. I could see how that could be useful, if you prompt within adetailer for the entire head.

Anzhc's Face Segmentation model is fantastic. It generates very good face masks. The results are far better than the square mask. I've been using this in several adetailer tests and getting very good results. The parameters I'm working with are (the only ones that even seem meaningful for mask quality):

1. erosion/dilation (seems to work better with some dilation because the generated face mask is within the outline of the face, and by expanding it then inpaint can cover over the outline, otherwise often a shadow is left from the previous face, another thing to consider is if the face is in profile their eyelashes will extend outside of the mask, generally 4 works well)
2. mask blur (zero blur actually works well since the face mask is so accurate, depends on the values of the other two parameters, I found at 64 it produced noise, 4 or 8 seems to work well, even 2)
3. masked padding (this one is confusing, all the articles of what this does simply say it adjusts the amount of padding, which isn't helpful – "padding" isn't defined which leads me to suspect the author doesn't understand this parameter, and the few articles I can find about inpainting are incomplete, where the author is guessing or doesn't know the meaning of this parameter, even the developer of inpainting doesn't document this parameter's effect.)

After experiments, masked padding seems to adjust how much of the area outside of the mask is used in creating the new image. Something like providing more context for the new image, where 0 would mean ignore everything outside of the mask and 256 means use that many pixels outside of the mask as context for the new image. Then, when the final image is blended, it's only within the mask area. None of the padding area is actually seen in the final image. Setting this to 0 produced the worst results. 128 generally produced better quality inpainting of faces. For example, hairlines and ears are better blended. Yet 256 also works in many cases.

Something that made this testing more difficult is how adetailer can save the mask previews, but it's the mask from the model not the resulting mask after applying adjustments. It would be very helpful to see the resulting mask. To try and determine this for myself I set the blur to 0, make adjustments, and then look at the before/after to trying and see what the mask might have been.

I should write an article for Civitai about using adetailer with Anzhc's Face Segmentation. I hesitate since I can't explain what some of the inpainting parameters really only do. I'm just guessing, and that's not the type of article I would want to write.

[Bing-su]

Thank you so much for your interest in this project.

In the Settings section of the Stable Diffusion, there is an option to save the mask image used for inpainting. This should help you.

The inpainting options in Adetailer are almost a direct copy of the inpainting options in stable-diffusion-webui. If you search for 'stable diffusion webui inpainting', you'll probably get several results. Some of the values are fixed in Adetailer, some are just not configurable (e.g. resize mode is 'Just resize', masked content is 'original').

I hope you find some of these videos and webpages I found helpful, thanks.

https://youtu.be/sF3POwPUWCE?si=3iouV_bLY6lkEXdM https://youtu.be/urNISRdbIEg?si=_Uiz82tpPqHZ68AQ https://openaijourney.com/stable-diffusion-inpainting-guide/