search

cubiq / ComfyUI_FaceAnalysis

Extension for ComfyUI to evaluate the similarity between two faces
Apache License 2.0
227 stars 25 forks source link

Underlying vs surface features of the face | comparison analyses #23

Closed abcnorio closed 3 weeks ago

abcnorio commented 3 weeks ago

Hello Matteo,

one remark on face comparison - if one uses e.g. the "zombie genre" (which is typically characterized by different flesh wounds, blood, strange or even empty eyes, and other distortions of the face) the "face distance" results in a high value which assumes more dissimilarity as actually is present. Assumed is that those additional "art features" of the face prevent the model to find high similarities, because the newly generated image (compared to the original one) with high prob won't show the same marks on the face. The zombie genre is just an extreme example, it may be the same with people who have traditional paintings/ piercings/ tattoos/ etc. on their faces that are not maintained in exactly that way after being processed by the model.

I need more examples to try out whether this is a constant effect, but wouldn't it make sense to implement something that can distinguish between facial features and "surface features"? Assumed is that would require a unique trained model that is able to learn the distinction between such underlying and surface located facial features. Is that a correct understanding of the situation - OR - does this not play a role and I just did not try out enough examples to find that out?

best

cubiq commented 3 weeks ago

in those cases always use "cosine", there's some pre-processing that we could do to maybe improve the result in those extreme cases but, cosine should give you a decent enough idea. of course the number will be pretty high, but as long as it's under 1.0 the person has been recognized

abcnorio commented 3 weeks ago

Thanks, will use that. Compared it to some other trials with "normal" faces and there the values were much better compared to such special cases of faces (which somehow supports my assumptions that it is not due to wrong configuration of the nodes and its parameters).