How to make background stable?

[burning846]

Hi, when I was trying to reimplement DFL by tensorflow 2.0, I found that the images my decoders generated had a bad background which didn't happened when I run the code in your repo. However, when I reviewed your code, it seemed that you didn't training the background(hair or something) as well when you set mask_training = True and bg_style_power=False, because I thought your code here ignored the background of the reconstruction image by multiplying the image and the mask:

gpu_target_src_masked_opt  = gpu_target_src*gpu_target_srcm_blur if masked_training else gpu_target_src
gpu_target_dst_masked_opt  = gpu_target_dst_masked if masked_training else gpu_target_dst

gpu_pred_src_src_masked_opt = gpu_pred_src_src*gpu_target_srcm_blur if masked_training else gpu_pred_src_src
gpu_pred_dst_dst_masked_opt = gpu_pred_dst_dst*gpu_target_dstm_blur if masked_training else gpu_pred_dst_dst

so I wonder how you make the background stable even if your didn't train the background, or did I miss something important?

Here is my settings

============ Model Summary ============
==                                   ==
==            Model name: test_SAEHD ==
==                                   ==
==     Current iteration: 60620      ==
==                                   ==
==---------- Model Options ----------==
==                                   ==
==            resolution: 256        ==
==             face_type: f          ==
==     models_opt_on_gpu: True       ==
==                 archi: dfhd       ==
==               ae_dims: 256        ==
==                e_dims: 64         ==
==                d_dims: 48         ==
==           d_mask_dims: 16         ==
==       masked_training: True       ==
==            learn_mask: True       ==
==             eyes_prio: True       ==
==            lr_dropout: False      ==
==           random_warp: True       ==
==             gan_power: 1.0        ==
==       true_face_power: 0.01       ==
==      face_style_power: 0.0        ==
==        bg_style_power: 0.0        ==
==               ct_mode: none       ==
==              clipgrad: True       ==
==              pretrain: False      ==
==       autobackup_hour: 0          ==
== write_preview_history: True       ==
==           target_iter: 0          ==
==           random_flip: True       ==
==            batch_size: 1          ==
==                                   ==
==----------- Running On ------------==
==                                   ==
==          Device index: 0          ==
==                  Name: TITAN RTX  ==
==                  VRAM: 23.65GB    ==
==                                   ==
=======================================

This is my output (eyes masked for privacy issue）:

This is the expected output from this repo:

reimplement environment

• Ubuntu 16.04.5 LTS
• TITAN RTX
• tensorflow 2.1.0

[burning846]

Ablation study shows that there is something wrong with gan training. When I set gan_power=0, the output seems similar to the image generated by this repo.