Error during video

[Nandu-Louro]

Hi, I need to get emotions from a video with a participant doing a task.

Your work seems really interesting and for the majority, it works. However, there are moments during the video where this error happens: Error raised during video mode. Traceback (most recent call last): File "main_esr9.py", line 253, in main() File "main_esr9.py", line 239, in main args.size, args.cuda, args.frames, args.branch, args.no_plot, args.face_detection) File "main_esr9.py", line 172, in video raise e File "main_esr9.py", line 159, in video gradcam) File "/Desktop/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks-master/controller/cvision.py", line 118, in recognize_facial_expression input_face = _pre_process_input_image(face) File "/Desktop/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks-master/controller/cvision.py", line 205, in _pre_process_input_image image = uimage.resize(image, ESR.INPUT_IMAGE_SIZE) File "/Desktop/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks-master/model/utils/uimage.py", line 268, in resize return cv2.resize(image, output_size) cv2.error: OpenCV(4.5.1) /tmp/pip-req-build-n_alixql/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

The program is able to ignore moments where the face is not present which is great, but there are situations where the participant looks around and this error happens.

Can you help me?

[siqueira-hc]

Dear,

Thank you for your message. The error seems to be raised from the OpenCV's resize method. Maybe, the input image is None or corrupted. If so, do not call the facial expression method.

Best, Henrique Siqueira

-- Henrique Siqueira Research Engineer Autonomous Driving at Huawei, Munich, Germany Doctoral Candidate at the University of Hamburg, Hamburg, Germany https://www.linkedin.com/in/siqueira-hc/ https://www.inf.uni-hamburg.de/en/inst/ab/wtm/people/siqueira.html

---

From: Nandu-Louro @.> Sent: Saturday, March 27, 2021 1:10 PM To: siqueira-hc/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks @.> Cc: Subscribed @.***> Subject: [siqueira-hc/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks] Error during video (#24)

Hi, I need to get emotions from a video with a participant doing a task.

Your work seems really interesting and for the majority, it works. However, there are moments during the video where this error happens: Error raised during video mode. Traceback (most recent call last): File "main_esr9.py", line 253, in main() File "main_esr9.py", line 239, in main args.size, args.cuda, args.frames, args.branch, args.no_plot, args.face_detection) File "main_esr9.py", line 172, in video raise e File "main_esr9.py", line 159, in video gradcam) File "/Desktop/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks-master/controller/cvision.py", line 118, in recognize_facial_expression input_face = _pre_process_input_image(face) File "/Desktop/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks-master/controller/cvision.py", line 205, in _pre_process_input_image image = uimage.resize(image, ESR.INPUT_IMAGE_SIZE) File "/Desktop/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks-master/model/utils/uimage.py", line 268, in resize return cv2.resize(image, output_size) cv2.error: OpenCV(4.5.1) /tmp/pip-req-build-n_alixql/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

The program is able to ignore moments where the face is not present which is great, but there are situations where the participant looks around and this error happens.

Can you help me?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/siqueira-hc/Efficient-Facial-Feature-Learning-with-Wide-Ensemble-based-Convolutional-Neural-Networks/issues/24, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGLDYDINWRTRCJXOYASEFMDTFXDKVANCNFSM4Z43JURA.

[Nandu-Louro]

in main_esr9.py after line 153 we have:

Video has been processed

        if img is None:
            break
        else:  # Process frame
            fer = None if (img is None) else cvision.recognize_facial_expression(img,
                                                                                 device,
                                                                                 face_detection,
                                                                                 gradcam)

This should deal with the image/frame error right? but how can I add to check if it is corrupted?

Thanks for your reply

[Nandu-Louro]

I got it, for some reason in cvision.py on recognize_facial_expression face = image[face_coordinates[0][1]:face_coordinates[1][1], face_coordinates[0][0]:face_coordinates[1][0], :] in certain frames results as [ ]

I added these lines right after the previous line and now the execption does not happen if face.size == 0: to_return_fer = FER(image) return to_return_fer