This is the ipadpter changed the material, what is the relationship with zeroshot

[wencxxxxxx]

This is the ipadpter changed the material, what is the relationship with zeroshot

[kealiu]

Add the paper & official demo code, it do use IPA to extract the materials. and also use depth to gather the depth information. And then, extract the illumination information by grey-out the target subject. by replace the original image with an illumination image, it combine the lighting and materials, give out a better image.

This node replace the original image to the illumination image (I-init image in the paper figure), and other function just use exist solutions.

[kealiu]

Thanks for this question, i update the README file.

As the paper figure, Image Encode indeed the IPA, Geometry Guidance indeed the Depth ControlNet.

This node complete the gap of Latent Illumination Guidance in the paper figure.

i hope this can answer your question.

[wencxxxxxx]

You mention "Extracting illumination information by graying the target object, replacing the original image with a light image." I don't think this is right, because at a noise reduction of 1, the light image can no longer affect the diffusion result. This is how I understand it

[kealiu]

my opinion, it add noise to it, but not simple mask out all the inpaint area.

When the original image of inpainting transfer to latent space, the maskout zone still have impact on the input latent.

you can change this Node's brighter input, and find out the changes of output.

[wencxxxxxx]

This is the function of nodes like the function of screenshots, am I right

[kealiu]

Yes! Gamma correction similar with the brighter.

[wencxxxxxx]

thanks