search

dchatel / comfyui_facetools

These custom nodes provide a rotation aware face extraction, paste back, and various face related masking options.
MIT License
45 stars 7 forks source link

Face mask for ona person on cuts. #9

Open Maelstrom2014 opened 3 weeks ago

Maelstrom2014 commented 3 weeks ago

HI!

Im making faceswaping and need to have only one face on mask.
Espesially its important when faces located near each other. I made it with making two step masking: First with CropFaces cropfactor: 1, mask Second with CropFaces cropfactor: 2, and inpaint resized (2 time smaller) mask to black mask. Can you add somehow option for (onlyone face mask making)?

ps: I use depthmap for contolnet, and it need more pix space outsize of face. (with cropfactor 1 depth-fm, best depth maker, not working at all) Cropfactor: 2 is important for inpainting to model understand better where and how face is located.

dchatel commented 3 weeks ago

Can you share your workflow, so I get a better idea of what you're trying to do ?

Regarding faces that are close to each others, it's best to either use the convex_hull mask or a combined mask between the convex_hull and either bisenet or johnathanidu if you need to take care of possible occlusions.

Maelstrom2014 commented 2 weeks ago

port_face2

here is a WF faceswap_send.json

dchatel commented 2 weeks ago

What is happening here is that the johnathanidu model is more precise than bisenet. It's expected, but johnathanidu model also takes more memory than bisenet. This should not be an issue if you don't do videos with johnathanidu, though.

Regarding bisenet, and this family of models in general, they are trained to segment a single face. So, this is an edge case you're having here. It's best to either use:

In both cases, the crop factor should not be a problem.

Maelstrom2014 commented 2 weeks ago

What is happening here is that the johnathanidu model is more precise than bisenet. It's expected, but johnathanidu model also takes more memory than bisenet. This should not be an issue if you don't do videos with johnathanidu, though.

Regarding bisenet, and this family of models in general, they are trained to segment a single face. So, this is an edge case you're having here. It's best to either use:

  • johnathanidu if it's enough to get a single face.
  • bisenet multiplied by convex_hull to get a single face and still have face occlusions.

In both cases, the crop factor should not be a problem.

No, I try to explain again. 1) Bisinet and johnathanidu works only good on images not bigger than 512. (bigger images - mask is degrading) 2) Both models mask more than one face, if cropping_factor face more than 1 (another face in frame) 3) Only way is my way step 1) make mask with cropping_factor=1. step 2) culculate again with cropping_factor=2. step 3) zoom out mask 1 to empty mask size 2. see picture how to eliminate two step culculation?