search

ltdrdata / ComfyUI-Impact-Pack

Custom nodes pack for ComfyUI This custom node helps to conveniently enhance images through Detector, Detailer, Upscaler, Pipe, and more.
GNU General Public License v3.0
1.89k stars 183 forks source link

Problem with fingers #393

Closed yaoleynikov closed 10 months ago

yaoleynikov commented 10 months ago

Hi, can you help me with fixing fingers.

Model: sdXL_v10VAEFix.safetensors

Screenshot 2023-12-22 180705

fingers.json

if-ai commented 10 months ago

Yes, I think it has a problem duplicating the pose estimation somewhere on the detailer image manipulation operation for example I made the estimation with the auxiliary nodes in the same image and I get two good hands https://i.imgur.com/yJGJD2C.png but if I use the segs I get two overlaping hands is overlaping the detectted left and right hand https://i.imgur.com/bJl4tQu.png

ltdrdata commented 10 months ago

This issue arises due to the limitations of the DWPose Preprocessor. If the DWPose Preprocessor fails to recognize the correct hand shape, it cannot fix it manually. If you wish to make corrections, you should apply the manually modified controlnet image instead of the output image from DWPose Preprocessor.

if-ai commented 10 months ago

This issue arises due to the limitations of the DWPose Preprocessor. If the DWPose Preprocessor fails to recognize the correct hand shape, it cannot fix it manually. If you wish to make corrections, you should apply the manually modified controlnet image instead of the output image from DWPose Preprocessor.

it is strange because trying with the controlnet preprocessor the same image and the hand is not duplicated on open pose. I included images done in the same image with different result using the same processor

ltdrdata commented 10 months ago

This issue arises due to the limitations of the DWPose Preprocessor. If the DWPose Preprocessor fails to recognize the correct hand shape, it cannot fix it manually. If you wish to make corrections, you should apply the manually modified controlnet image instead of the output image from DWPose Preprocessor.

it is strange because trying with the controlnet preprocessor the same image and the hand is not duplicated on open pose. I included images done in the same image with different result using the same processor

It's not a big issue if duplication occurs. Only the masked regions are involved in image reconstruction. The crucial point is that the structure of landmarks composing the hand needs to be correctly positioned.

if-ai commented 10 months ago

Alright, I am sure you know way better than me what is happening and what is right. Thank you so much for the reply and all the amazing work on your nodes and developments you share.

yaoleynikov commented 10 months ago

This issue arises due to the limitations of the DWPose Preprocessor. If the DWPose Preprocessor fails to recognize the correct hand shape, it cannot fix it manually. If you wish to make corrections, you should apply the manually modified controlnet image instead of the output image from DWPose Preprocessor.

Can you show how I can fix that? Because almost all my knowledge based on your tutorials!

ltdrdata commented 9 months ago

This issue arises due to the limitations of the DWPose Preprocessor. If the DWPose Preprocessor fails to recognize the correct hand shape, it cannot fix it manually. If you wish to make corrections, you should apply the manually modified controlnet image instead of the output image from DWPose Preprocessor.

Can you show how I can fix that? Because almost all my knowledge based on your tutorials!

To achieve this, you would need to use something like OpenPose Editor. It can be quite cumbersome. Rather than that, try exploring an approach that leverages the recently released MeshGraphormer.

https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/workflow/meshgraphomer_dwpose_tile-segs-handfix.png

https://www.reddit.com/r/comfyui/comments/19dlbp2/hands_fix_meshgraphormer_impactpack/