xaviviro / refacer

Refacer: One-Click Deepfake Multi-Face Swap Tool
MIT License
520 stars 127 forks source link

Crashes after the 1st few frames #9

Closed eliasts closed 1 year ago

eliasts commented 1 year ago

After the latest tensorrt update it crashes with the following errors:

gradio message: OpenCV(4.7.0) :-1: error: (-5:Bad argument) in function 'warpAffine' > Overload resolution failed: > - M is not a numpy array, neither a scalar > - Expected Ptr for argument 'M'

Console message: Total frames: 1204 Extracting frames: 100%|█████████████████████████████████████████████████████████| 1204/1204 [00:00<00:00, 1585.98it/s] Processing frames: 2%|█▎ | 26/1204 [00:01<01:18, 15.05it/s] Traceback (most recent call last): File "K:\AI-APPS\refacer\venv\lib\site-packages\gradio\routes.py", line 427, in run_predict output = await app.get_blocks().process_api( File "K:\AI-APPS\refacer\venv\lib\site-packages\gradio\blocks.py", line 1323, in process_api result = await self.call_function( File "K:\AI-APPS\refacer\venv\lib\site-packages\gradio\blocks.py", line 1051, in call_function prediction = await anyio.to_thread.run_sync( File "K:\AI-APPS\refacer\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "K:\AI-APPS\refacer\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "K:\AI-APPS\refacer\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, args) File "K:\AI-APPS\refacer\app.py", line 30, in run return refacer.reface(video_path,faces) File "K:\AI-APPS\refacer\refacer.py", line 171, in reface results = list(tqdm(executor.map(self.process_faces, frames), total=len(frames),desc="Processing frames")) File "K:\AI-APPS\refacer\venv\lib\site-packages\tqdm\std.py", line 1178, in iter for obj in iterable: File "C:\Users\elias\AppData\Local\Programs\Python\Python310\lib\concurrent\futures_base.py", line 621, in result_iterator yield _result_or_cancel(fs.pop()) File "C:\Users\elias\AppData\Local\Programs\Python\Python310\lib\concurrent\futures_base.py", line 319, in _result_or_cancel return fut.result(timeout) File "C:\Users\elias\AppData\Local\Programs\Python\Python310\lib\concurrent\futures_base.py", line 458, in result return self.get_result() File "C:\Users\elias\AppData\Local\Programs\Python\Python310\lib\concurrent\futures_base.py", line 403, in __get_result raise self._exception File "C:\Users\elias\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\thread.py", line 58, in run result = self.fn(self.args, **self.kwargs) File "K:\AI-APPS\refacer\refacer.py", line 130, in process_faces faces = self.__get_faces(frame) File "K:\AI-APPS\refacer\refacer.py", line 125, in get_faces face.embedding = self.rec_app.get(frame, kps) File "K:\AI-APPS\refacer./recognition\arcface_onnx.py", line 65, in get aimg = face_align.norm_crop(img, landmark=kps, image_size=self.input_size[0]) File "K:\AI-APPS\refacer./recognition\face_align.py", line 72, in norm_crop warped = cv2.warpAffine(img, M, (image_size, image_size), borderValue=0.0) cv2.error: OpenCV(4.7.0) :-1: error: (-5:Bad argument) in function 'warpAffine'

Overload resolution failed:

  • M is not a numpy array, neither a scalar
  • Expected Ptr for argument 'M'

Windows 10 nvidia 3060 12GB Python 3.10.9 Latest Tensorrt

xaviviro commented 1 year ago

Thank you for reporting this issue. In order to better diagnose the problem, could you please provide more information? Specifically, could you share the size of the video you're trying to process and the size of the images you've uploaded as faces?Also, could you please specify which ProcessExecutor the program is picking up? Is it TensorrtExecutionProvider or CUDAExecutionProvider?

I apologize for the inconvenience, I'm aware that the error handling and messaging need improvements. Your feedback is extremely valuable in this regard.

I want to add that initially, my focus was on functionality. I have recently been working on performance enhancements. Now, I'm about to shift my focus to better error handling. While I work on identifying and resolving this issue, I would suggest trying out the application on Google Colab, if possible. This might offer a temporary workaround. Thank you for your understanding and patience.

xaviviro commented 1 year ago

While we identify and resolve the issue, you can also use the CPU mode with the —force_cpu flag.

Use this command:

python app.py —force_cpu
eliasts commented 1 year ago

Video size: 640x352

src image size: 512x583 dst image size: 512x512

TENSORRT mode with providers ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']

xaviviro commented 1 year ago

While we have not yet resolved the bug, we have disabled TensorrtExecutionProvider. It should now automatically use CUDAExecutionProvider and should work for you. Could you let us know if this is the case? At the moment we don't have an NVIDIA GPU to test on

eliasts commented 1 year ago

With CUDAExecutionProvider it works!