TJ16th / comfyUI_TJ_NormalLighting

Custom Node for comfyUI for virtual lighting based on normal map
139 stars 9 forks source link

looks like a fun effect. Just some thoughts on it. #9

Open CommunicationDesignation opened 5 months ago

CommunicationDesignation commented 5 months ago

I wouldn't be able to do this, not without a lot of time learning, I know enough to get around. You made a node to make videos of the light move, do you think a node to preview the Light position on the image node, a live update using the cursor position as the light source, a click or something would set the position. an image preview with a live update would be better then hitting the generate button each time a variable is changed. using the cursor method to pick the light position aside it would be just as good having the image update without generating a new image every-time a variable is changed. I haven't tried your node yet so I don't know if it already does this.

something else, I think having some way to effect the non lit areas and treat them as shadows. the normal map image canvas always fills every pixel so I would think that means it contains more than just where the light hits. From what I've read normal maps cannot, or at least never are used for shadows. in 3d normal maps and depth maps are used in combination for light and shadow. Maybe the reason why normal maps cant do shadows or in other words only data from the image that applies to what would be the highlights of an image. It would mean changing the operation that processes normal map image data, I don't know what that operation would be called.

I wondered if normal maps could be converted to 3d mesh. I learned normal map can be converted to a displacement map which can be used to create actual depth on a 3dmesh and that normal maps are a way to create fake depth. Light-field screens like the lume pad made by Leia inc. have a pseudo "holographic" screen. Its a depth that was only found with special with only 3d glasses, but with this screen you don't need 3d glasses, better than Nintendo 3ds. the things on screen not only look like they pop off the screen but a depth that goes into and through the device. There's more but im getting off track, like google has done an App on the lume pad will make existing images into light-field pictures (impressive but pseudo 3d) and it does this by creating a depth map from the image and then some AI and other magic. Point is, normal, depth, displacement, a combination of them all, or something else, is their a way to add in shadows. A solution would be to have 2 images, the original and a darkened one and the normal would be the highlight. that would only work with layers and segmented masks and not always worth the effort. The only reason this would be helpful is if someone wanted to change the original lighting of the image, AI generated or not. The godot engine uses normal maps and shaders for light and dark on 2d games. Its a neat effect.

One last thing. I would like to commission or suggest a custom node idea to people that offer those services. where are some places to look if you wanted to talk to someone about creating comfy nodes?