how to enhancing the working time of verify and find function

[bhushanjha716]

i am working on a project which uses the face recognition on a large dataset i am using deepface library for it but its taking so much time to do in youtube videos you show that way but in new version of deepface no option for it can you answer me @serengil

[serengil]

Cannot understand the problem. Be more clear please.

[bhushanjha716]

i working on a project called employee attendance with face recognition i am using deepface for face recognition ,when register a new employee it checks in database if this employee registered or not when it check it takes so much time

[bhushanjha716]

https://www.youtube.com/watch?v=Hrjp-EStM_s in this video you showed how to work large scale data but in new version these things are not available

[bhushanjha716]

import os
import pandas as pd
import numpy as np
from datetime import datetime
import csv
from deepface import DeepFace
import base64
import cv2
def bs64_to_frame(bs64):
        decoded_bytes = base64.b64decode(bs64)
        decoded_image = np.frombuffer(decoded_bytes, dtype=np.uint8)
        decoded_image = cv2.imdecode(decoded_image, flags=cv2.IMREAD_COLOR)
        return decoded_image

def register(bs64,id,name,dept):
    known_ids=[]
    check2=[]
    count=False
    path='dataset'
    excelDataSet='Employee_details.csv'
    l1 = [os.path.join(path, f) for f in os.listdir(path) if f.endswith('.jpg')]
    frame=bs64_to_frame(bs64)
    if frame is not None:
        for i in l1:
            known_ids.append(os.path.splitext(os.path.basename(i))[0])
            check=DeepFace.verify(i,frame,model_name='Facenet',normalization='Facenet',detector_backend='dlib')
            print(check)
            check2.append(check['verified'])
        if True in check2:
            return 'your data is already in dd!'
        else:
            if id in known_ids:
                return 'your data is already in dd!!!!!'
            else:     
                count=True
                cv2.imwrite(path+f'/{id}.jpg',frame)  
        if not os.path.exists(excelDataSet):
            with open(excelDataSet,'w') as ff:
                csveriting = csv.writer(ff)
                csveriting.writerow(['EmployeeID','Emp_name','Department'])
                csveriting.writerow([id,name,dept])
        else:
            with open(excelDataSet,'a') as ff:
                csvreading=pd.read_csv(excelDataSet)
                csveriting = csv.writer(ff)
                for i in range(len(csvreading)):
                    if csvreading['EmployeeID'].values[i]==id and count==True:
                        return 'your details already in d'

                    else:
                        csveriting.writerow([id,name,dept])
                        break
        df = pd.read_csv(excelDataSet)
        df.to_csv(excelDataSet, index=False)
        print(df)
    else:
        return 'Unable to process image'

    return str("Save") 
``` `

[serengil]

That is an old video. You need to call verify function in a for loop.

[bhushanjha716]

i am calling verify function in for loop at line 26 , on five to six images its taking around 15 seconds thats to much @serengil

[serengil]

Add this somewhere before for loop. Verify will take shorter.

_ = DeepFace.build_model("Facenet")

[bhushanjha716]

import time
start_time = time.time()
import os
import pandas as pd
import numpy as np
from datetime import datetime
import csv
from deepface import DeepFace
import base64
import cv2

def bs64_to_frame(bs64):
    decoded_bytes = np.frombuffer(base64.b64decode(bs64), dtype=np.uint8)
    return cv2.imdecode(decoded_bytes, flags=cv2.IMREAD_COLOR)

def register(bs64, id, name, dept):
    known_ids = []
    path = 'dataset'
    excel_dataset = 'Employee_details.csv'

    # Image decoding optimization
    frame = bs64_to_frame(bs64)
    if frame is None:
        return 'Unable to process image'

    # Resize for efficiency (adjust size as needed)
    frame = cv2.resize(frame, (224, 224))
    _=DeepFace.build_model('Facenet')

    # Efficient face verification loop with early stopping
    l1 = [os.path.join(path, f) for f in os.listdir(path) if f.endswith('.jpg')]
    for i in l1:
        known_ids.append(os.path.splitext(os.path.basename(i))[0])
        check = DeepFace.verify(i, frame, model_name='Facenet', normalization='Facenet', detector_backend='mtcnn')
        print(check['time'])
        if check['verified']:
            return 'Your data is already in!'

    # Check if ID exists in known IDs efficiently (advanced)
    with open(excel_dataset, 'r') as f:
        csv_reader = csv.reader(f)
        if id in (row[0] for row in csv_reader):  # Assuming ID is in the first column
            return 'Your data is already in!'

    # Save registration details
    cv2.imwrite(path + f'/{id}.jpg', frame)

    if not os.path.exists(excel_dataset):
        with open(excel_dataset, 'w') as ff:
            csv_writer = csv.writer(ff)
            csv_writer.writerow(['EmployeeID', 'Emp_name', 'Department'])
            csv_writer.writerow([id, name, dept])
    else:
        with open(excel_dataset, 'a') as ff:
            csv_writer = csv.writer(ff)
            csv_writer.writerow([id, name, dept])

    print(f"Employee data saved for ID: {id}")
    return "Registration successful!"

After adding that line you said above , this is the output below: 1/1 ━━━━━━━━━━━━━━━━━━━━ 1s 717ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 173ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 44ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 63ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 48ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 47ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 50ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 42ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 55ms/step 3/3 ━━━━━━━━━━━━━━━━━━━━ 0s 92ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 222ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 55ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 39ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 41ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 41ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 45ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 49ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 35ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 38ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 219ms/step 9.41 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 102ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 61ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 43ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 44ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 42ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 49ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 41ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 31ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 40ms/step 16/16 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 48ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 40ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 44ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 41ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 44ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 40ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 52ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 42ms/step 4.57 Your data is already in! Execution time: 39.4513 seconds PS C:\Users\bhushan\Desktop\face_recognition system (2)> Please consider it seriously @serengil

[serengil]

if your concern is speed, try default detector instead of mtcnn.

check = DeepFace.verify(i, frame, model_name='Facenet')

that progress bar is coming from mtcnn!

[bhushanjha716]

i already do that not so much change here is the output with default detector : 10.64 1.77 Your data is already in! Execution time: 42.3273 seconds PS C:\Users\bhushan\Desktop\face_recognition system (2)> i saw your that video i mentioned above that logic is very nice that type of something can you tell me @serengil

[serengil]

How many images are you processing for that time?

[bhushanjha716]

around six to seven

[serengil]

In that video, find function is being used not verify. You cannot make this faster until you invest a more powerful hardware.

[bhushanjha716]

in new version of deepface find function doesn't have posintional argument for model which is you done that in your video

[serengil]

you do not have to set model anymore in find function, just set model_name string.

[bhushanjha716]

in other device i run it on gpu but in that device its take around 15 seconds

[serengil]

okay, with find function, you will have much faster results.

[bhushanjha716]

you do not have to set model anymore in find function, just set model_name string.

i does but that is not faster anymore in your video it takes less than 2 seconds and here its taking so much time

[serengil]

in your first run, yes it is slow. but in second run, if the pickle is ready, then it should be much faster. I can find an identity in 100 faces in just 1-2 seconds.

[bhushanjha716]

in my project around 60 to people images are there in dataset and if this that so much time than my project is not so efficient please do something

[serengil]

how long find function takes? 1st time and 2nd time results please.

[bhushanjha716]

find function duration 36.67322540283203 seconds 1st time it takes this find function duration 5.9738850593566895 seconds 2nd time it takes this

[serengil]

Did you put this before find function?

_ = DeepFace.build_model("Facenet")

Also, are you using opencv for find function?

[bhushanjha716]

yes i am doing that before find function and opencv as a detector backend?

[serengil]

so, that is all what you can do. you cannot decrease time more.

[bhushanjha716]

@serengil thanks sir for your concern its so helpful ,you replied many times thank you for your time