Inpainting color change

[imperator-maximus]

Hello,

I tried the hi-res fix example using Inpainting: https://comfyanonymous.github.io/ComfyUI_examples/2_pass_txt2img/

The inpainting result looks good but original pixels outside mask are pixelated which also makes sense because I downscaled 1024 image to 512 made the Inpainting and used Latent Upscale coming back to 1024. The whole workflow looks very similar but I used normal non inpainting model + Inpainting VAE Encode + non inpainting model for second sampler. This is mask and result:

To solve the issue I added LatentCompositionMasked node which might be same as "apply_overlay" parameter in Diffuser project and got this result:

This looks not so bad (beside the masked edges at the lower left but my mask is quite large - so this is just as it is). Now comes the problem the whole image gets an drastic color change here putting it in original context:

This is a known problem other servers have but overall it is solved more than a year ago. So I am wondering why there isn't any example showing hires fix+ Inpainting on 1.x at all for ComfyUI in whole internet. Thank you.

[HelloClyde]

You might need to use the ImageCompositeMasked node to overlay the original image on the non-masked parts, and the mask could use some feathering.

[imperator-maximus]

You might need to use the ImageCompositeMasked node to overlay the original image on the non-masked parts, and the mask could use some feathering.

yes that is what I tried first but results had been more worse

[HelloClyde]

I guess it might have something to do with the model capacity and VAE? I tried it and it wasn't very exaggerated.

[imperator-maximus]

I made 1024x1024 and yours is 768 but this does not matter. Yes you have same color change in your example which is a show-stopper:

I am not that deep an AI programmer to find out what is wrong here but it would be nice having an official working example here because this is more an quite old "standard" functionality and not a test of some exotic new crazy AI.

[HelloClyde]

My understanding is that both Hires Fix and inpainting are generated through diffusion models, and even with the use of the LatentCompositionMasked node, the final result will definitely be different from the original image (in terms of color, details, etc.). That's why the last step requires the use of ImageCompositeMasked to overlay the non-masked parts. ImageCompositeMasked is purely an image overlay (similar to layer overlay in Photoshop), which ensures that the non-masked parts are exactly the same as the original image.

[imperator-maximus]

ok I made more tests and removed HiResFix but problem is still the same just using the standard Inpainting example: edges around the mask have color change or edges at the whole image are wrong. So I guess 1.5+Inpainting just does not work.

[DaviSMoura]

@imperator-maximus any news in fixing it?

[imperator-maximus]

@imperator-maximus any news in fixing it?

yes these nodes do this job! Color change is even better than SDXL standard inpainting (it darkens the color a bit at SDXL)

[imperator-maximus]

1.5 seems to work fine now. Will check SDXL options later.