Acly / comfyui-inpaint-nodes

Nodes for better inpainting with ComfyUI: Fooocus inpaint model for SDXL, LaMa, MAT, and various other tools for pre-filling inpaint & outpaint areas.
GNU General Public License v3.0
576 stars 41 forks source link

QOL features for InpaintWithModel #14

Closed spacepxl closed 6 months ago

spacepxl commented 6 months ago
Acly commented 6 months ago

Looks good, thanks.

Some kind of tiling is probably a better way to get high resolution output from inpaint models, but a bit more work. It didn't seem that important if it's only used as pre-fill for SD. What made you add the upscale model?

spacepxl commented 6 months ago

If it's just used as a pre-fill for SD, it probably makes no difference, but for cases where the inpaint model result is good enough without SD, having an upscaling model can improve the quality of textures/details compared to just bilinear upscaling. It's fine if you don't want to include that, that's why I separated out the commits. Maybe it would be better to name that input so it's more obvious that it's not required.

I don't know much about the architecture of the inpaint models, but I'm guessing that tiling wouldn't be an easy thing to add. Maybe an auto-crop to region of interest + padding for large images with only a small inpaint patch? Also, this is off-topic, but it would be awesome to add support for the MAT FFHQ-512 model (for faces).

Acly commented 6 months ago

By tiling I mean splitting the input into crop boxes with some overlap, running the inpaint model on each of them, and merging them back together with some blending for the overlap region. Similar to comfy.utils.tiled_scale with scale=1. But yes it requires some experimentation, I'm not even sure about those size restrictions (MAT seems to be ok with higher resolutions, but not lower than 512, while Lama just generates grey mush at some point). I'm also fine with merging it as it is.

Maybe it would be better to name that input so it's more obvious that it's not required.

This sounds like a good idea