FPS issue for the real-time experinece in Webcam

[sandhyacs]

Hi, Can you tell me how can i increase it's Fps as it is only 0.87 right now which is not good for the real time experience.

[arnaldog12]

That's weird. Are you running on CPU or GPU? What's the frame size you're capturing from webcam?

[sandhyacs]

Running on GPU...frame size is 640x480.

[arnaldog12]

Let's continue our investigation then:

• What's the architecture of your model?
• How many parameters it has?

Can you share the code you're using to make predictions from webcam?

[sandhyacs]

Yeah sure I will share...

On Sat, 16 Nov, 2019, 8:01 PM Arnaldo Gualberto, notifications@github.com wrote:

Let's continue our investigation then:

• What's the architecture of your model?
• How many parameters it has?

Can you share the code you're using to make predictions from webcam?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/arnaldog12/Deep-Learning/issues/3?email_source=notifications&email_token=AE2MXTPALIULQXADPIMIVWTQT772XA5CNFSM4JNWL3YKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEHSYZI#issuecomment-554642533, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2MXTJOID6OHRZJZZCOST3QT772XANCNFSM4JNWL3YA .

[sandhyacs]

here is the code for the webcam prediction:

import cv2 import dlib import numpy as np import matplotlib.pyplot as plt import _pickle as pkl from threading import Thread from keras.models import Sequential, load_model from keras.layers import Dense from keras.callbacks import EarlyStopping from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler

def detect_face_points(image): detector = dlib.get_frontal_face_detector() predictor = dlib.shape_predictor("models/landmarks.dat") face_rect = detector(image, 1) if len(face_rect) != 1: return []

dlib_points = predictor(image, face_rect[0])
face_points = []
for i in range(68):
    x, y = dlib_points.part(i).x, dlib_points.part(i).y
    face_points.append(np.array([x, y]))       
return face_points

print(face_points)

def compute_features(face_points): assert (len(face_points) == 68), "len(face_points) must be 68"

face_points = np.array(face_points)
features = []
for i in range(68):
    for j in range(i+1, 68):
        features.append(np.linalg.norm(face_points[i]-face_points[j]))

return np.array(features).reshape(1, -1)

cap = cv2.VideoCapture(0) model = load_model('models/model.h5') while(True):

Capture frame-by-frame

ret, frame = cap.read()

# Our operations on the frame come here
im = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
face_points = detect_face_points(im)
for x, y in face_points:
    cv2.circle(im, (x, y), 1, (0, 255, 0), -1)
features = compute_features(face_points)
y_pred = model.predict(features)
roll_pred, pitch_pred, yaw_pred = y_pred[0]
print(' Roll: {:.2f}°'.format(roll_pred))
print('Pitch: {:.2f}°'.format(pitch_pred))
print('  Yaw: {:.2f}°'.format(yaw_pred))
# Display the resulting frame
cv2.imshow('frame',im)
if cv2.waitKey(1) & 0xFF == ord('q'):
    break

When everything done, release the capture

cap.release() cv2.destroyAllWindows()

[arnaldog12]

The bug is clear now. You're loading the dlib's frontal_face_detector and shape_predictor every time before detecting a face and its landmarks. They are very time-consuming resources to load. Take them out of detect_face_points method (maybe load them together with the model) and try again. This may fix the low-fps problem.

Please, close the issue if it solved your problem. Thanks!

[sandhyacs]

Thank you so much @arnaldog12 for your help. Yeah, it increased fps at some extent. but still i have to increase it more.