[Issue]: dev - issue with Face restore (in combination with Time Steps, it seems)

[Tillerz]

Issue Description

When creating an image with face restoration ON and also time steps ON, face restoration produces a weird shadowy artifact-prone face (and also modifies other parts of the image, not sure if that happened before). When I switch time steps to OFF, this does not happen.

Version Platform Description

WSL-2, Windows11, Chrome, version 0680a88

Face Restoration OFF:

Face Restoration ON (see the bugged out face):

Face Restoration ON, but Time Steps OFF (no face distortion happening):

Relevant log output

no log output that looks weird/different

Prompt: vivid, colorful, intrinsic details, 1woman, portrait of a cute and beautiful 30 years old noble italian woman with long auburn_hair, skimpy regency era miniskirt, offshoulder crop top, (very low overflowing neckliine:1.4), (indoors, noble victorian regency era style villa:1.1), beautiful shoulders, (long gorgeous narrow hips:1.2), (perky bulging overflowing boobs:1.2), watercolor, portraits, very soft skin, beautiful legs

Negative: (advertisement:1.3), (girl, child, loli, kid, teen:1.2), username, signature, logo, text, (obese, fat, overweight, wide hips, big boobs:1.4), marvel, disney

Parameters: Steps: 40| Seed: 486962143| Sampler: DPM++ 2M| CFG scale: 3.5| Size: 512x768| Parser: Full parser| Model: zo4-ra-art-if-noo-fcb3-rcr| Model hash: 0512113be6| Backend: Diffusers| App: SD.Next| Version: 0680a88| Comment: | Operations: hires; face; txt2img; upscale| Second pass: True| Hires force: True| Hires steps: 20| Hires upscaler: ESRGAN 4x Remacri| Hires upscale: 2| Hires resize: 0x0| Hires size: 1024x1536| Denoising strength: 0.3| Hires sampler: Same as primary| Image CFG scale: 3| CFG rescale: 0.7| Face restoration: Face HiRes| FreeU: b1=1.2 b2=1.4 s1=0.9 s2=0.2| ADetailer model: female_breast-v4.2.pt| ADetailer confidence: 0.3| ADetailer dilate erode: 4| ADetailer mask blur: 4| ADetailer denoising strength: 0.4| ADetailer inpaint only masked: True| ADetailer inpaint padding: 32| ADetailer version: 24.5.1| Sampler options: sde-dpmsolver++/order 2/low order| Pipeline: StableDiffusionPipeline| HiDiffusion Ratios: 0.2/0.1| HiDiffusion: True/True/True:8| Size-1: 512| Size-2: 768| Hires resize-1: 0| Hires resize-2: 0| Hires size-1: 1024| Hires size-2: 1536

Backend

Diffusers

Branch

Dev

Model

SD 1.5

Acknowledgements

• [X] I have read the above and searched for existing issues
• [X] I confirm that this is classified correctly and its not an extension issue

[vladmandic]

there is no magic to face-hires, its an inpaint process of a region of a image and it uses same settings used to generate actual image. if you think that it makes image worse, can you enable settings -> image options -> save image before face restoration and upload same image before and after face-hires?

[Tillerz]

As discussed on discord, seems the negative prompt weights make it bug out.