[!IMPORTANT] This Repository is no longer maintained (see this comment). Recommend using the up-to-date Fork instead.

sd-forge-ic-light

A1111/SD Forge extension for IC-Light. Forge backend is based on https://github.com/huchenlei/ComfyUI-IC-Light-Native.

[2024-06-06] Major Update [PR #24]

• Load models from ic-light folder with arbitrary filenames. You need to rename your unet folder to ic-light.
• Use the pre-built rembg package instead of diffusers version.
• New Feature: Implement Difference of Gaussians to reintroduce some details, e.g. text, after the processing.
• New Feature: Implement reinforce-fg option which allows better preservation of fg base color.

Getting Started

0. For Automatic1111, install sd-webui-model-patcher first
1. Download the two models from Releases
2. Create a new folder, ic-light, inside your webui models folder
3. Place the 2 models inside the said folder
4. (Optional) You can rename the models, as long as the filenames contain either fc or fbc

How to use

For best result, it is recommended to use low CFG and strong denosing strength.

Given FG, Generate BG and relight [Txt2Img][HR available]

Infotext:

sunshine from window, a beautiful girl, absurdres, highres, (masterpiece:1.2), (best quality, highest quality),
Negative prompt: (lowres, low quality, worst quality:1.2), (text:1.2), watermark, (frame:1.2), deformed, ugly, deformed eyes, blur, out of focus, blurry, deformed cat, deformed, photo, anthropomorphic cat, monochrome, photo, pet collar, gun, weapon, blue, 3d, drones, drone, buildings in background, green
Steps: 25, Sampler: DPM++ 2M, Schedule type: Karras, CFG scale: 2, Seed: 3615474644, Size: 512x768, Model hash: e6415c4892, Model: realisticVisionV20_v20, Clip skip: 2, Version: v1.9.3-13-g8e355f08

Given FG and light map, Genereate BG and relight [Img2Img]

Img2Img input image is lightmap. After you select value from the radio, the img2img input will automatically be set by the extension. You can also upload your own lightmap by selecting Custom.

Infotext:

sunshine from window, a beautiful girl, absurdres, highres, (masterpiece:1.2), (best quality, highest quality),
Negative prompt: (lowres, low quality, worst quality:1.2), (text:1.2), watermark, (frame:1.2), deformed, ugly, deformed eyes, blur, out of focus, blurry, deformed cat, deformed, photo, anthropomorphic cat, monochrome, photo, pet collar, gun, weapon, blue, 3d, drones, drone, buildings in background, green
Steps: 20, Sampler: DPM++ 2M, Schedule type: Karras, CFG scale: 2, Seed: 2984374687, Size: 512x768, Model hash: e6415c4892, Model: realisticVisionV20_v20, Denoising strength: 0.96, Clip skip: 2, Version: v1.9.3-13-g8e355f08

Given FG and BG, Put FG on BG and relight [Txt2Img][HR available]

Infotext:

a beautiful girl, absurdres, highres, (masterpiece:1.2), (best quality, highest quality),
Negative prompt: (lowres, low quality, worst quality:1.2), (text:1.2), watermark, (frame:1.2), deformed, ugly, deformed eyes, blur, out of focus, blurry, deformed cat, deformed, photo, anthropomorphic cat, monochrome, photo, pet collar, gun, weapon, blue, 3d, drones, drone, buildings in background, green
Steps: 25, Sampler: DPM++ 2M, Schedule type: Karras, CFG scale: 2, Seed: 2230971215, Size: 512x768, Model hash: e6415c4892, Model: realisticVisionV20_v20, Clip skip: 2, Version: v1.9.3-13-g8e355f08

[2024-06-06] Restore Detail

Detail transfer was originally implemented in https://github.com/kijai/ComfyUI-IC-Light. It captures high frequency details, e.g. text, in the input fg image and reapplys them to the output image.

Original output:

After detail restore:

Infotext:

A bottle of oyster sauce, kitchen counter, absurdres, highres, (masterpiece:1.2), (best quality, highest quality),
Negative prompt: (lowres, low quality, worst quality:1.2), (text:1.2), watermark, (frame:1.2), deformed, ugly, deformed eyes, blur, out of focus, blurry, deformed cat, deformed, photo, anthropomorphic cat, monochrome, photo, pet collar, gun, weapon, blue, 3d, drones, drone, buildings in background, green
Steps: 20, Sampler: DPM++ 2M, Schedule type: Karras, CFG scale: 2, Seed: 2984374687, Size: 768x768, Model hash: e6415c4892, Model: realisticVisionV20_v20, Denoising strength: 0.96, Clip skip: 2, Version: v1.9.3-13-g8e355f08

[2024-06-06] Reinforce FG

A big problem of IC-Light is that it often alters the FG object's base color too much. By adding the fg image on top of the lightmap, this issue can be alleviated. This essentially implements this comfyui workflow. Here is a comparison:

Without reinforce-fg: You can observe that the fg object almost looks transparent.

With reinforce-fg (Same seed): The fg object no longer look transparent.

Skip remove bg

If the default remove bg cannot achieve your desired effect, you can use other tools to create an RGBA image and uncheck the remove bg checkbox. Image with grey background can also be used the same way as RGBA image.

Known Issue

• Restore Details does not work properly when the input and output aspect ratios are different