search

deepfakes / faceswap

Deepfakes Software For All
https://www.faceswap.dev
GNU General Public License v3.0
52.08k stars 13.19k forks source link

crash on extract faces from a mp4 video #1120

Closed 524c closed 3 years ago

524c commented 3 years ago

crash_report.2021.01.27.204515980430.log

01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:04 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:05 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:06 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:07 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:07 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:07 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:07 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:07 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:07 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:08 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:09 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:10 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:11 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:12 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:13 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:14 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:15 MainProcess align_cv2-dnn_aligner_input_0 multithreading run DEBUG Error in thread (align_cv2-dnn_aligner_input_0): OpenCV(4.5.1) /private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/pip-req-build-39p1qqfs/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'\n 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Setting thread 01/27/2021 20:45:15 MainProcess MainThread image _set_thread DEBUG Thread pre-exists and is alive: <lib.multithreading.MultiThread object at 0x14bb2c1f0> 01/27/2021 20:45:15 MainProcess MainThread multithreading check_and_raise_error DEBUG Thread error caught: [(<class 'cv2.error'>, error("OpenCV(4.5.1) /private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/pip-req-build-39p1qqfs/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'\n"), <traceback object at 0x150127500>)] Traceback (most recent call last): File "/Users/user1/Desktop/faceswap/lib/cli/launcher.py", line 182, in execute_script process.process() File "/Users/user1/Desktop/faceswap/scripts/extract.py", line 119, in process self._run_extraction() File "/Users/user1/Desktop/faceswap/scripts/extract.py", line 216, in _run_extraction for idx, extract_media in enumerate(status_bar): File "/Users/user1/.pyenv/versions/p38/lib/python3.8/site-packages/tqdm/std.py", line 1166, in iter for obj in iterable: File "/Users/user1/Desktop/faceswap/plugins/extract/pipeline.py", line 237, in detected_faces if self._check_and_raise_error(): File "/Users/user1/Desktop/faceswap/plugins/extract/pipeline.py", line 660, in _check_and_raise_error if plugin.check_and_raise_error(): File "/Users/user1/Desktop/faceswap/plugins/extract/_base.py", line 347, in check_and_raise_error err = thread.check_and_raise_error() File "/Users/user1/Desktop/faceswap/lib/multithreading.py", line 84, in check_and_raise_error raise error[1].with_traceback(error[2]) File "/Users/user1/Desktop/faceswap/lib/multithreading.py", line 37, in run self._target(*self._args, **self._kwargs) File "/Users/user1/Desktop/faceswap/plugins/extract/_base.py", line 465, in _thread_process batch = function(batch) File "/Users/user1/Desktop/faceswap/plugins/extract/align/_base.py", line 255, in _process_input result = self.process_input(batch) File "/Users/user1/Desktop/faceswap/plugins/extract/align/cv2_dnn.py", line 54, in process_input faces, batch["roi"], batch["offsets"] = self.align_image(batch) File "/Users/user1/Desktop/faceswap/plugins/extract/align/cv2_dnn.py", line 82, in align_image face = cv2.resize(face, dsize=sizes, interpolation=interpolation) cv2.error: OpenCV(4.5.1) /private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/pip-req-build-39p1qqfs/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

============ System Information ============ encoding: UTF-8 git_branch: master git_commits: 2208eb3 Bugfix: Edge case extracted face sub-cropping error. da5f693 Add python3.8-tk to Dockerfile.gpu and fixed documentation regarding docker launch (#1118). 15bbc95 Update Dockerfile (#1112). 163b8fe sysinfo/setup - Standardize cuda/cudnn checks. f9a5b72 Bugfix - setup.py. Explicitly install cudatoolkit in Conda gpu_cuda: No global version found gpu_cudnn: No global version found gpu_devices:
gpu_devices_active:
gpu_driver: 0 gpu_vram:
os_machine: x86_64 os_platform: macOS-11.2-x86_64-i386-64bit os_release: 20.3.0 py_command: faceswap.py extract -i /Users/user1/Desktop/face11.mp4 -o /Users/user1/Desktop/faces/face1 py_conda_version: N/A py_implementation: CPython py_version: 3.8.7 py_virtual_env: True sys_cores: 8 sys_processor: i386 sys_ram: Total: 16384MB, Available: 6275MB, Used: 8436MB, Free: 18MB

=============== Pip Packages =============== absl-py==0.11.0 appnope==0.1.2 astunparse==1.6.3 backcall==0.2.0 cachetools==4.2.1 certifi==2020.12.5 cffi==1.14.4 chardet==4.0.0 cycler==0.10.0 decorator==4.4.2 enum34==1.1.10 face-alignment==1.3.3 face-recognition-models==0.3.0 fastcluster==1.1.26 ffmpy==0.2.3 flatbuffers==1.12 gast==0.3.3 google-auth==1.24.0 google-auth-oauthlib==0.4.2 google-pasta==0.2.0 grpcio==1.32.0 h5py==2.10.0 idna==2.10 imageio==2.9.0 imageio-ffmpeg==0.4.3 ipython==7.19.0 ipython-genutils==0.2.0 jedi==0.18.0 joblib==1.0.0 Keras==2.2.4 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.2 kiwisolver==1.3.1 llvmlite==0.35.0 Markdown==3.3.3 matplotlib==3.2.2 networkx==2.5 numba==0.52.0 numpy==1.18.5 nvidia-ml-py3 @ git+https://github.com/deepfakes/nvidia-ml-py3.git@6fc29ac84b32bad877f078cb4a777c1548a00bf6 oauthlib==3.1.0 opencv-python==4.5.1.48 opt-einsum==3.3.0 parso==0.8.1 pathlib==1.0.1 pexpect==4.8.0 pickleshare==0.7.5 Pillow==8.1.0 plaidml==0.7.0 plaidml-keras==0.7.0 prompt-toolkit==3.0.14 protobuf==3.14.0 psutil==5.8.0 ptyprocess==0.7.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycparser==2.20 Pygments==2.7.4 pynvx==1.0.0 pyparsing==2.4.7 python-dateutil==2.8.1 PyWavelets==1.1.1 PyYAML==5.4.1 requests==2.25.1 requests-oauthlib==1.3.0 rsa==4.7 scikit-image==0.18.1 scikit-learn==0.24.1 scipy==1.6.0 six==1.15.0 tensorboard==2.2.2 tensorboard-plugin-wit==1.8.0 tensorflow==2.2.2 tensorflow-estimator==2.2.0 termcolor==1.1.0 threadpoolctl==2.1.0 tifffile==2021.1.14 tqdm==4.56.0 traitlets==5.0.5 typing-extensions==3.7.4.3 urllib3==1.26.2 wcwidth==0.2.5 Werkzeug==1.0.1 wrapt==1.12.1

================= Configs ================== --------- convert.ini ---------

[color.color_transfer] clip: True preserve_paper: True

[color.match_hist] threshold: 99.0

[color.manual_balance] colorspace: HSV balance_1: 0.0 balance_2: 0.0 balance_3: 0.0 contrast: 0.0 brightness: 0.0

[writer.pillow] format: png draw_transparent: False optimize: False gif_interlace: True jpg_quality: 75 png_compress_level: 3 tif_compression: tiff_deflate

[writer.ffmpeg] container: mp4 codec: libx264 crf: 23 preset: medium tune: none profile: auto level: auto skip_mux: False

[writer.gif] fps: 25 loop: 0 palettesize: 256 subrectangles: False

[writer.opencv] format: png draw_transparent: False jpg_quality: 75 png_compress_level: 3

[mask.mask_blend] type: normalized kernel_size: 3 passes: 4 threshold: 4 erosion: 0.0

[mask.box_blend] type: gaussian distance: 11.0 radius: 5.0 passes: 1

[scaling.sharpen] method: none amount: 150 radius: 0.3 threshold: 5.0

--------- gui.ini ---------

[global] fullscreen: False tab: extract options_panel_width: 30 console_panel_height: 20 icon_size: 14 font: default font_size: 9 autosave_last_session: prompt timeout: 120 auto_load_model_stats: True

--------- .faceswap --------- backend: cpu

--------- extract.ini ---------

[global] allow_growth: False

[detect.mtcnn] minsize: 20 threshold_1: 0.6 threshold_2: 0.7 threshold_3: 0.7 scalefactor: 0.709 batch-size: 8

[detect.cv2_dnn] confidence: 50

[detect.s3fd] confidence: 70 batch-size: 4

[align.fan] batch-size: 12

[mask.unet_dfl] batch-size: 8

[mask.vgg_obstructed] batch-size: 2

[mask.vgg_clear] batch-size: 6

--------- train.ini ---------

[global] centering: face coverage: 68.75 icnr_init: False conv_aware_init: False optimizer: adam learning_rate: 5e-05 reflect_padding: False allow_growth: False mixed_precision: False convert_batchsize: 16

[global.loss] loss_function: ssim mask_loss_function: mse l2_reg_term: 100 eye_multiplier: 3 mouth_multiplier: 2 penalized_mask_loss: True mask_type: extended mask_blur_kernel: 3 mask_threshold: 4 learn_mask: False

[model.realface] input_size: 64 output_size: 128 dense_nodes: 1536 complexity_encoder: 128 complexity_decoder: 512

[model.dfl_sae] input_size: 128 clipnorm: True architecture: df autoencoder_dims: 0 encoder_dims: 42 decoder_dims: 21 multiscale_decoder: False

[model.unbalanced] input_size: 128 lowmem: False clipnorm: True nodes: 1024 complexity_encoder: 128 complexity_decoder_a: 384 complexity_decoder_b: 512

[model.dlight] features: best details: good output_size: 256

[model.villain] lowmem: False

[model.dfaker] output_size: 128

[model.original] lowmem: False

[model.dfl_h128] lowmem: False

[trainer.original] preview_images: 14 zoom_amount: 5 rotation_range: 10 shift_range: 5 flip_chance: 50 disable_warp: False color_lightness: 30 color_ab: 8 color_clahe_chance: 50 color_clahe_max_size: 4

S-Hun commented 3 years ago

I also found a solution to this problem. It's not a perfect solution, but we've identified a error with file _"faceswap/plugins/extract/align/cv2dnn.py" and we've fixed it as below.

Edit function align_image in file:cv2_dnn.py: 

face = cv2.resize(face, dsize=sizes, interpolation=interpolation)

to

try:
    face = cv2.resize(face, dsize=sizes, interpolation=interpolation)
except:
    face = np.zeros((128, 128, 3), np.uint8)
    pass

When we resize, I handled the problem by replacing it with a blank image, and we could see that it was running again, although it was not the exact method.

torzdf commented 3 years ago

I think this is a memory error.

Sadly the cv2-dnn method is really not good, so never really going to prioritize looking at it. One day I may well research a better CPU friendly aligner (if anyone knows of any implementation/papers, then feel free to ping them over).