Recognition.py : inconsistent pickle file

[AndreaLanfranchi]

Pickle file holding representations for files in directory is named after model_name and detector_backend. After an initial pass to extract representations from all the images in the directory, assuming no new files have been added, no file have been changed the representations are loaded directly from the pickle file (without undergoing once again through detection and recognition of each face)

However the find not only receives as arguments the model and the detector (which, when changed, change also the pickle file name) but also the normalization argument which has an impact on the generated representations.

As a result if I call an initial

 find(
    [...]
    model_name="VGG-Face",
    detector_backend="opencv",
)

A pickle file holding representations with base technique is generated. If after that I call, say,

 find(
    [...]
    model_name="VGG-Face",
    detector_backend="opencv",
    normalization="Facenet"
)

then no representation is recomputed and the results of the find function may be highly unexpected.

[serengil]

Good spot. If we add the normalization name into the pickle file, then this will be sorted easily.

similarly, expand_percentage, align arguments have an impact on representations but these are also discarded, too. these should be added into the file name.

[AndreaLanfranchi]

`expand_percentage" does not have an impact on representations as those are constrained by fixed size input by each model. I don't think is relevant.

More relevant instead the "align" argument.

[serengil]

that is another bug. we should consider expand_percentage while we are finding bulk embeddings here - https://github.com/serengil/deepface/blob/master/deepface/modules/recognition.py#L173

[AndreaLanfranchi]

that is another bug. we should consider expand_percentage while we are finding bulk embeddings here - https://github.com/serengil/deepface/blob/master/deepface/modules/recognition.py#L173

I don't think. Expansion of a facial area doesn't have any impact on representation. But as you wish ... I only signalled a condition. Won't spend more time on this.

[serengil]

Expand percentage has definitely an impact on embeddings.

from deepface import DeepFace

img_path = "target.jpg"

DeepFace.represent(img_path=img_path, model_name="Facenet")[0]["embedding"][0:10]
DeepFace.represent(img_path=img_path, model_name="Facenet", expand_percentage=10)[0]["embedding"][0:10]

You can see the representation of same image with different expand_percentage values.

[0.47962263226509094,
 1.0006622076034546,
 1.0271168947219849,
 -0.6997617483139038,
 0.5685179233551025,
 -0.8650573492050171,
 1.301469087600708,
 0.4709436297416687,
 0.6177820563316345,
 1.1736094951629639]

[0.07209590077400208,
 1.220704197883606,
 1.0250455141067505,
 -1.0584845542907715,
 0.7029904723167419,
 -0.8850963711738586,
 0.9492990970611572,
 0.7604453563690186,
 0.7631895542144775,
 0.7458571791648865]

[AndreaLanfranchi]

I'll try to eplain: for me the usage and the meaning of expand_percentage is totally off and it should not even apply. There are multiple reasons:

1. Each detector has its own ways to define a face bounding box: some more squared some more rectangular. But they have one thing in common: the boundaries of the bounding box are exactly what are meant to be the boundaries of a detected face
2. expand_dims increases the boundaries of such facial area potentially including portions of the image which have nothing to do with the face
3. embeddings are always computed on a fixed size (each model has its own) input shape which means two things: 1) the wider (the more expanded facial area) the more CPU costs to resize the image and 2) the more "noise" around the face the more the embeddings get apart

Expand percentage has definitely an impact on embeddings.

Of course ... if you expand the facial area the resized input for the model receives a "smaller face" hence different embeddings Long story short : imo expand_percentage introduces a distortion argument which tampers with detections and embeddings. I only think it could be useful for visualization purposes : hence I completely removed it in my fork and won't reason again on it.

[serengil]

Closed with PR - https://github.com/serengil/deepface/pull/1185