Quality of results

[belladoreai]

Hey, big thanks for all your work! I think a node like this could be super useful, especially when filtering a large quantity of generated images when building datasets for characters. Unfortunately the quality of results appears to be bad at the moment. As an experiment, I took 4 high quality images of Charlize Theron, cropped them into squares, and used your FaceAnalysis node to compute the similarity between the faces. I was hoping to see low EUC values - low as in way lower than 0.3 - when comparing images of the same persons' face. But the EUC values I see are like 0.4 and 0.5. I also noticed that comparing the exact same photo to itself yields EUC values like 0.04 or 0.06 depending on the image. I would have expected that to be at 0.00 or like 0.01 at most.

If you have ideas on how to improve, it would be really cool to get this working correctly.

theron_comparison.json

[belladoreai]

I tried resizing all images to 512x512. Now the same image compared to itself yields 0 EUC as it should. However, the comparisons between images are still giving values between 0.4 to 0.6 for the same person.

[cubiq]

this is a very complex topic and I can't really go too deep into it right now but maybe I will post some documentation if there's interest.

There are many factors at play here. You should not take the result in an absolute context but relative to a certain generation. Head position, color, saturation, makeup can all alter the result. Especially the EUCL value. But to evaluate the result relative to a set of results or results from similar checkpoints, this could be a very useful tool.

From a face recog point of view 0.6 or less EUCL or 0.07 COS means that they are the same person. So in your example all results passed despite being very different shots.

To improve the kind of results in your example it would be better to send a set of images as reference and average the embeds. At the moment is not supported but it's next thing that I'm gonna do. But again that is not strictly what this extension is about.

Also at the moment I'm using DLib, I'll also add insighface and possibly others that should grant possibly more accurate results, but for the purpose of this extension (again relative comparison) this is pretty good.

[belladoreai]

From a face recog point of view 0.6 or less EUCL or 0.07 COS means that they are the same person. So in your example all results passed despite being very different shots.

Here's a counterexample of 2 completely different persons getting 0.6 EUCL:

I tried a bunch of random image pairs that represented different people, and FWIW I was not able to find anything that returned below 0.6, but I did get find several pairs of slightly above 0.6. Most results were between 0.65 and 0.80.

[cubiq]

you got 0.083 COS which it very high (btw EUC is above 0.6). There's a reason why I offer both EUC and COS, I may actually add other options.

[andupotorac]

@belladoreai Did you manage to get this to be more accurate?

[belladoreai]

@andupotorac Unfortunately no. And in general, I have not found any useful quantitative measure of "does the person in this photo look like the person in this other photo".

[andupotorac]

@andupotorac Unfortunately no. And in general, I have not found any useful quantitative measure of "does the person in this photo look like the person in this other photo".

Did you try this also? https://github.com/chflame163/ComfyUI_FaceSimilarity

[belladoreai]

I haven't tried that one. If you have, let me know how well it works.

[andupotorac]

I haven't tried that one. If you have, let me know how well it works.

Pending for me, as I have issues with the installation.