Blurry output

[epicstar7]

I have installed the project on windows and its working, but I noticed while my input images are both high resolution images, the swapped output face is quite blurry. It doesn't look similar to the example images. Is this a known issue, or is it is a matter of configuration? Any thoughts would be helpful.

[epicstar7]

There are also 2 errors shown, maybe this gives a clue:

sberswap\lib\site-packages\torch\nn\functional.py:3000: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and uses scale_factor directly, instead of relying on the computed output size. If you wish to keep the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details. warnings.warn("The default behavior for interpolate/upsample with float scale_factor changed "

and

sberswap\lib\site-packages\kornia\augmentation\augmentation.py:1833: DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead. category=DeprecationWarning,

[threefoldo]

These warnings are irrelevant. To improve the image quality, you can enable 'use_sr' option which will call face_enhancement. However, the pix2pix super resolution is not good, I'm working on an alternative model.

[epicstar7]

I tried use_sr but it didn't make a huge difference. Still somewhat blurry and suboptimal.

What alternative model are you looking at, something like GPEN/GFPGAN?

[AlexanderGroshev]

Hi, @epicstar7! The output face may look blurry because our model only generates a 256x256 resolution. To solve this problem, you can look at the face enhancement function. If you need a stronger super resolution model, check out sber-swap implementation with the GFPGAN sr model SberSwapInferenceGFPGAN.ipynb. It scales the output images up to 512x512 and the faces look more detailed. However, in this case, the inference time is longer.

[epicstar7]

Thanks a lot @AlexanderGroshev, let me try it. Is there a way to run this code from the conda commandline instead of a ipynb script file? I'm not really sure how to translate it to the right commands on there.

[AlexanderGroshev]

@epicstar7 check inference_GFPGAN.py:

python inference_GFPGAN.py --source_paths examples/images/beckham.jpg --target_video examples/videos/nggyup.mp4 --use_sr True

You need to clone GFPGAN repository, download weights GFPGANCleanv1-NoCE-C2.pth and place to GFPGAN/experiments/pretrained_models/ folder

[epicstar7]

This works mate, amazing job, this is a great improvement to this repo. Thanks!

[epicstar7]

I do get an error CUDA out of memory when trying to swap on video. I'm curious to know why this happens for videos while for single images there is no issue. This shouldn't happen right?

[AlexanderGroshev]

@epicstar7, try to reduce batch size, it might help.

[epicstar7]

This works, changed it with -batch_size 16 and now no CUDA memory error any more. But the face is not swapped in the output video!

[epicstar7]

Managed to get it working using a targetimage of the face. I noticed though swapping is sometimes failing after just a few frames in the video. You see the swapped face for a few frames and then it just returns to the original pictures. Any thoughts why this happens?