search

iperov / DeepFaceLab

DeepFaceLab is the leading software for creating deepfakes.
GNU General Public License v3.0
46.27k stars 10.38k forks source link

Video output is shaking. Link to video and merge settings in description #5716

Open oo92 opened 11 months ago

oo92 commented 11 months ago

Hi.

These are my merging settings:

Loading new_SAEHD model...

Initializing models: 100%|#############################################################################################################################################################################################################################| 4/4 [00:02<00:00,  1.43it/s]
======================== Model Summary ========================
==                                                           ==
==            Model name: new_SAEHD                          ==
==                                                           ==
==     Current iteration: 2422811                            ==
==                                                           ==
==---------------------- Model Options ----------------------==
==                                                           ==
==              use_fp16: False                              ==
==            resolution: 320                                ==
==             face_type: wf                                 ==
==     models_opt_on_gpu: True                               ==
==                 archi: liae-ut                            ==
==               ae_dims: 448                                ==
==                e_dims: 102                                ==
==                d_dims: 102                                ==
==           d_mask_dims: 22                                 ==
==       masked_training: True                               ==
==    retraining_samples: False                              ==
==            mouth_prio: False                              ==
==           uniform_yaw: False                              ==
==         blur_out_mask: True                               ==
==             adabelief: True                               ==
==            lr_dropout: y                                  ==
==         loss_function: SSIM                               ==
==           random_warp: False                              ==
==      random_hsv_power: 0.0                                ==
==     random_downsample: False                              ==
==          random_noise: False                              ==
==           random_blur: False                              ==
==           random_jpeg: False                              ==
==         random_shadow: none                               ==
==      background_power: 0.0                                ==
==       true_face_power: 0.0                                ==
==      face_style_power: 0.0                                ==
==        bg_style_power: 0.0                                ==
==               ct_mode: none                               ==
==          random_color: False                              ==
==              clipgrad: True                               ==
==              pretrain: False                              ==
==               cpu_cap: 8                                  ==
==       preview_samples: 4                                  ==
==    force_full_preview: False                              ==
==                    lr: 5e-05                              ==
==          session_name: lrd on                             ==
==       autobackup_hour: 12                                 ==
==     maximum_n_backups: 6                                  ==
== write_preview_history: False                              ==
==           target_iter: 0                                  ==
==       random_src_flip: False                              ==
==       random_dst_flip: True                               ==
==            batch_size: 8                                  ==
==             gan_power: 0.0                                ==
==        gan_patch_size: 44                                 ==
==              gan_dims: 16                                 ==
==         gan_smoothing: 0.1                                ==
==             gan_noise: 0.0                                ==
==       eyes_mouth_prio: False                              ==
==                                                           ==
==----------------------- Running On ------------------------==
==                                                           ==
==          Device index: 0                                  ==
==                  Name: NVIDIA GeForce RTX 3060 Laptop GPU ==
==                  VRAM: 4.18GB                             ==
==                                                           ==
===============================================================
[y] Use interactive merger? ( y/n ) : n
Choose mode: 
(0) original
(1) overlay
(2) hist-match
(3) seamless
(4) seamless-hist-match
(5) raw-rgb
(6) raw-predict
[1] : 4
[255] Hist match threshold ( 0..255 ) : 255

Choose mask mode: 
(0) full
(1) dst
(2) learned-prd
(3) learned-dst
(4) learned-prd*learned-dst
(5) learned-prd+learned-dst
(6) XSeg-prd
(7) XSeg-dst
(8) XSeg-prd*XSeg-dst
(9) learned-prd*learned-dst*XSeg-prd*XSeg-dst
[1] : 4

[0] Choose erode mask modifier ( -400..400 ) : 2020
[0] Choose blur mask modifier ( 0..400 ) : 100100
[0] Choose motion blur power ( 0..100 ) : 0
[0] Choose output face scale modifier ( -50..50 ) : 0
Color transfer to predicted face ( rct/lct/mkl/mkl-m/idt/idt-m/sot-m/mix-m ) : 

Choose sharpen mode: 
(0) None
(1) box
(2) gaussian
[0] ( ?:help ) : 0

[0] Choose super resolution power ( 0..100 ?:help ) : 0
[0] Choose image degrade by denoise power ( 0..500 ) : 0
[0] Choose image degrade by bicubic rescale power ( 0..100 ) : 0
[0] Degrade color power of final image ( 0..100 ) : 0

[16] Number of workers? ( 1-16 ?:help ) : 16

This is my video: https://drive.google.com/file/d/1GuJ9ONTKuLqCSfThHW8DqI8-2bgHZzJ1/view?usp=sharing

I've trained the model until loss stopped decreasing and this is what I got.

Elsaam2y commented 10 months ago

I see the color is not consistent. Did you try setting the color transfer parameter? Can you also share the ist and src videos to better tell where is the problem?

oo92 commented 10 months ago

@Elsaam2y

I select rct for color transfer.

This is data_src: https://drive.google.com/drive/folders/1NUsGGAmE0vSZdiDy23c3DUhALqpuGE9P?usp=sharing

This is data_dst: https://drive.google.com/file/d/1HMm5-wDgr7qvtj0Pqvdd7_5lTxYyfWJq/view?usp=sharing

Elsaam2y commented 10 months ago

Okay I See. I think it works fine considering the technology limitation especially with inconsistent lightning, like in your dst video. This could be the problem why you are getting this inconsistent color distribution in the final output. To be more sure, I would encourage you to train using a simpler dst video with fixed lightning.