search

yuval-alaluf / hyperstyle

Official Implementation for "HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing" (CVPR 2022) https://arxiv.org/abs/2111.15666
https://yuval-alaluf.github.io/hyperstyle/
MIT License
998 stars 115 forks source link

the generated image is a little blur #46

Closed tengshaofeng closed 2 years ago

tengshaofeng commented 2 years ago

tbq Thanks for your great work. It give me the best performance as far as I know for reconstruction of real image. But I found the reconstruct image is a litte blur. The left of upload image is real image, and the right is the reconstructed one. How can I get more clear image? They are all 1024x1024 without resizing from 256 to 1024.

tbq the left of the next image is aligned and resize to 1024x1024, and the right is reconstructed one.

yuval-alaluf commented 2 years ago

Blurry images can sometimes occur because we are altering the actual StyleGAN generator so some artifacts could occur. One thing I've found that helps reduce the blurriness is reducing the number of steps used for inference. This isn't a perfect solution but it does help a bit from my experience. One other solution that could lead to less blurriness is trying to incorporate a new loss to the training of HyperStyle to try to reduce the blurriness of the reconstructions.

tengshaofeng commented 2 years ago

Thanks. By the way，How to define the new loss.

发自我的iPhone

------------------ Original ------------------ From: yuval-alaluf @.> Date: Sat,Jul 2,2022 11:36 PM To: yuval-alaluf/hyperstyle @.> Cc: bojohn @.>, Author @.> Subject: Re: [yuval-alaluf/hyperstyle] the generated image is a little blur (Issue #46)

Blurry images can sometimes occur because we are altering the actual StyleGAN generator so some artifacts could occur. One thing I've found that helps reduce the blurriness is reducing the number of steps used for inference. This isn't a perfect solution but it does help a bit from my experience. One other solution that could lead to less blurriness is trying to incorporate a new loss to the training of HyperStyle to try to reduce the blurriness of the reconstructions.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

McFredward commented 2 years ago

I also experience this kind of blurriness with a non-facial domain trained Hyperstyle network. I guess that's the biggest issue with this model right now, because beside this, the model is surprisingly good at recreating fine details!

@yuval-alaluf So I would be also interested how you would adapt the loss to punish blur images (Iterating the generator only once during inference made no visible difference for me).

tengshaofeng commented 2 years ago

Another question. I found the performance is not good for smile of face editor. 飞书20220420-101513_org

origin image, reconsturcted image, simle , laugh The reconsturcted image is almost same as the origin image. And it can really control smiling, but it is not good look, for example the teeth is ugly. How can I get a better performance? thank you.

yuval-alaluf commented 2 years ago

Regarding the blurriness, I believe one possible loss could be something like a frequency-based loss in order to preserve the higher frequencies and prevent blurriness? Perhaps a discriminator-based loss could also assist here? I have not tried these myself, but from experience, these could possibly help the sharpness of the images.

yuval-alaluf commented 2 years ago

How can I get a better performance? thank you. You could possibly try decreasing the number of iterations to perform the inversion or constrain which layers are altered during the tuning process. This could help keep the model in a good place with respect to the original model

tengshaofeng commented 2 years ago

This could help keep the model in a good place w Thank you for your reply and advise.