OpenTalker / video-retalking

[SIGGRAPH Asia 2022] VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild
https://opentalker.github.io/video-retalking/
Apache License 2.0
6.66k stars 978 forks source link

mask_sharp = cv2.GaussianBlur #136

Closed BryanHarrisScripts closed 1 year ago

BryanHarrisScripts commented 1 year ago

Error

File "inference.py", line 345, in main() File "inference.py", line 201, in main pred, , = enhancer.process(img, img, face_enhance=True, possion_blending=False) File "C:\Users\bryan\video-retalking\third_part\GPEN\gpen_face_enhancer.py", line 113, in process mask_sharp = cv2.GaussianBlur(mask_sharp, (0,0), sigmaX=1, sigmaY=1, borderType = cv2.BORDER_DEFAULT) UnboundLocalError: local variable 'mask_sharp' referenced before assignment

What is the resolution that it is looking for??

Found in Logs:

"In my experience, when the face resolution is too large or the face area proportion is too large, it may not be detected. This may be related to the face detector's training set."

Jeriousman commented 1 year ago

@BryanHarrisScripts Did you make it work? If so, how did you do that?

BryanHarrisScripts commented 1 year ago

The issue stemmed from two factors: 1. the image was upscaled, resulting in excessive resolution; 2. there was motion in the video. I believe these were the problems, as once I reduced the resolution using cv2's resize function, and slowed down the motion using Premiere Pro and keyframes, the issue was resolved.

import cv2 video = cv2.VideoCapture('C:/users/bryan/desktop/test.mp4') width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT)) print(f'The video is {width} pixels wide and {height} pixels tall.')

Jeriousman commented 1 year ago

@BryanHarrisScripts Thank you very much. Will consider that! And thank you for the checking code too.

AIhasArrived commented 1 year ago

The issue stemmed from two factors: 1. the image was upscaled, resulting in excessive resolution; 2. there was motion in the video. I believe these were the problems, as once I reduced the resolution using cv2's resize function, and slowed down the motion using Premiere Pro and keyframes, the issue was resolved.

import cv2 video = cv2.VideoCapture('C:/users/bryan/desktop/test.mp4') width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT)) print(f'The video is {width} pixels wide and {height} pixels tall.')

Hello, how do you "slow down the motion" using premiere pro and frames please? I am not sure I understand. Thanks