Questions on using DeepFace.find() but with a custom face-detection

[Terminazor]

For my use case (a live-cam-analysis-setting) I have a face detection model already, but still need a good face-recognition model and I'm using DeepFace.find(): DeepFace.find(face_image, db_path = "./train_cut_faces/", model_name = model_name, model=model, enforce_detection = False, silent =True) as "model" ["VGG-Face", "Facenet", "Facenet512", "OpenFace", "DeepFace", "DeepID", "ArcFace", "Dlib", "SFace"] are used (I want to test them all for my usecase) and have some questions:

1.) When using enforce_detection = False, do the images in db_path should only contain the face? (I guess this must be true) 2.) The creation of my face-database is done in a different script (lets call it training.py), so, do I need to somehow (pre-)process the faces (for example with cv2.resize(face_image, (224, 224)) or normalize/standardize the faces in a certain way? 3.) If 2.) is not needed, should the stored faces have a minimum/maximum pixel-size? The face detection algorithm used can detect faces up to a few meters from the camera and some faces have like 20x40 pixels 4.) When DeepFace.find() is called the first time it creates a representations_deeface.pkl file with the embeddings of the faces in db_path (as far as I understand) and reuses the .pkl if it was already created. In my usecase the user can add persons/faces and the name to the database at will, is it possible to somehow "update" the .pkl? 5.) What is in your opinion the "best" "middle-ground" model for a live-analysis setting in terms of accuracy and speed? Due to this setting at least a few FPS is desirable. 6.) Does the choice of the distance_metric function has a significant impact on the recognized faces?

Thank you for your time!

[serengil]

1- if you set enforce_detection = False then neither images in folder or current image do not have to have face.

2- we do not have a face database in deepface. find function stores the embeddings for images in the folder in a pickle file. before finding embedding, it applies resize step automatically.

3- no max limit. they will be resized to the target size. e.g. 224x224 for vgg.

4- update is not currently supported. you have to delete the pickle if you want to update it. i plan to add updating functionality in the next releases.

5- facenet or arcface. they are complex to build but once you build it using it again again is easy.

6- of course

[Risingabhi]

Hi @serengil while working with some images here, i am not sure why function "findCosinedistance" from file functions.py gives strange outputs. sending 2 images here: changes i made are : I am calculating embeddings directly by using the "represent" function, and then feeding them to findCosine.....

earlier i was getting accurate results , but today, its giving me weird outputs. check 2 images attached for your reference.

0.2793804645640111 -- from findCosineDistance for these images.

My present code:
`img1 = cv2.imread("C:/Users/Risin/Desktop/Students/Student/1002/1012/AdarshKumar_72_1.jpg")

img2 = cv2.imread("C:/Users/Risin/Desktop/Students/Student/1002/1012/Subash_68_1.jpg")

embeddings = represent(img1, model_name='VGG-Face', model= None, enforce_detection= False,
                               detector_backend='mtcnn', align=True, normalization='VGGFace')

embedding2 = represent(img2, model_name='VGG-Face', model= None, enforce_detection= False,
                               detector_backend='mtcnn', align=True, normalization='VGGFace')

rsult = findCosineDistance(embeddings,embedding2)
print(rsult)`

[serengil]

1- pass img_path to represent function directly

embeddings = represent(img_path = "C:/Users/Risin/Desktop/Students/Student/1002/1012/AdarshKumar_72_1.jpg"
, model_name='VGG-Face', enforce_detection= False, detector_backend='mtcnn')

2- do not use normalization

3- why cosine is weird? if it is less than 0.40, then they are same person. (Ref: https://github.com/serengil/deepface/blob/master/deepface/commons/distance.py)

[Terminazor]

@serengil Regarding 4): I have looked through the code and found the part where the embeddings are calculated and the .pkl file is stored. I put it all into a function and it seems to do the job. The only thing I added was the removal of the .pkl file right before saving it again with the new embeddings:

def update_database(db_path, model_name ='VGG-Face', distance_metric = 'cosine', model = None, enforce_detection = True, detector_backend = 'opencv', align = True, prog_bar = True, normalization = 'base', silent=False):

    if os.path.isdir(db_path) == True:

        # Load or build model
        if model == None:

            if model_name == 'Ensemble':
                if not silent: print("Ensemble learning enabled")
                models = Boosting.loadModel()

            else: #model is not ensemble
                model = build_model(model_name)
                models = {}
                models[model_name] = model

        else: #model != None
            if not silent: print("Already built model is passed")

            if model_name == 'Ensemble':
                Boosting.validate_model(model)
                models = model.copy()
            else:
                models = {}
                models[model_name] = model

        #---------------------------------------

        if model_name == 'Ensemble':
            model_names = ['VGG-Face', 'Facenet', 'OpenFace', 'DeepFace']
            metric_names = ['cosine', 'euclidean', 'euclidean_l2']
        elif model_name != 'Ensemble':
            model_names = []; metric_names = []
            model_names.append(model_name)
            metric_names.append(distance_metric)

        #---------------------------------------

        file_name = "representations_%s.pkl" % (model_name)
        file_name = file_name.replace("-", "_").lower()

        #find representations for db images
        employees = []
        representations = []
        pbar = tqdm(range(0,len(employees)), desc='Finding representations', disable = prog_bar)

        #for employee in employees:
        for index in pbar:
            employee = employees[index]

            instance = []
            instance.append(employee)

            for j in model_names:
                custom_model = models[j]

                representation = represent(img_path = employee
                    , model_name = model_name, model = custom_model
                    , enforce_detection = enforce_detection, detector_backend = detector_backend
                    , align = align
                    , normalization = normalization
                    )

                instance.append(representation)

            #-------------------------------

            representations.append(instance)

        if path.exists(db_path+'/'+file_name):
            os.remove(db_path+'/'+file_name)

        f = open(db_path+'/'+file_name, "wb")
        pickle.dump(representations, f)
        f.close()

        if not silent: print("Representations stored in ",db_path,"/",file_name," file. This file has been updated!")

    else:
        raise ValueError("Passed db_path does not exist!")

[Risingabhi]

1- pass img_path to represent function directly

embeddings = represent(img_path = "C:/Users/Risin/Desktop/Students/Student/1002/1012/AdarshKumar_72_1.jpg"
, model_name='VGG-Face', enforce_detection= False, detector_backend='mtcnn')

2- do not use normalization

3- why cosine is weird? if it is less than 0.40, then they are same person. (Ref: https://github.com/serengil/deepface/blob/master/deepface/commons/distance.py)

Refer to point 3 by you: when i run this, at times i get different person, as Identified. I am extracting face images by using MTCNN() and then calculating its vector (VGG-Face) . check the verificiation result of first 5 images.