Novel-StyleGAN-Inversion-Papers

Share some intersting paper about StyleGAN, especially StyleGAN Inversion.

The ultimate goal is to revise my paper Collaborative Learning for Faster StyleGAN Embedding, and then release the code.

• Why I need to revise my paper? I think I should hack into the details of GAN Inversion, such as the influence of interpolating (due to the different size of input and output images), the design of the encoder... What's more, I found there is no standard split to evaluate the inversion effect.

The pioneer in StyleGAN Inversion.

1. Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
2. StyleGAN Encoder. [code1] [code2]
3. Image2StyleGAN++: How to Edit the Embedded Images?

Image2StyleGAN is the first paper in StyleGAN inversion. In this work, they conducted some interesting experiments which is meaningful to the future works. StyleGAN Encoder is another pioneer work, which takes the results of a nerual network as initial point of the next optimization produre. StyleGAN Encoder has two representative repos, in which code1 is ealier and code2 has better inversion effect. Image2StyleGAN++ was expanded to optimize both latent vector and the noise maps.

TODO

• [x] Reproduce Image2StyleGAN
• [ ] Reproduce Image2StyleGAN++.
  • I won't reimplement Image2StyleGAN++ right now.

A comprehensive survey

1. GAN Inversion: A Survey. github This paper comprehensively reviewed current papers of StyleGAN inversion.

How to design the encoder? These papers give their solutions.

1. Designing an Encoder for StyleGAN Image Manipulation.
2. Encoding in style: a stylegan encoder for image-to-image translation.
3. ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement.
   • ReStyle slacks the one-step forward pass to multi-step learning. This idea makes sense, which somewhat shares same spirt with my paper. My question is whether the Fig.6 can indicates general phenomenon.

How to constraint the gan inversion?

1. Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation.[github]
   • The discriminator-based constraint and progressively training policy are interesting.
2. LOHO: Latent Optimization of Hairstyles via Orthogonalization
   • how to design constraints on hair? this paper gives an answer.

Application

1. In&Out : Diverse Image Outpainting via GAN Inversion
   • Outpainting is an interesting application by patch-based inversion.
   • The prior loss part again point out how to ensure the inversed latent code staying in the gan domain.
2. Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing. [github]
   • First training an encoder to inverse images. Then combine the guidence image and the source image in the feature-level to realize local editting effect.
3. InfinityGAN: Towards Infinite-Resolution Image Synthesis
   • novel work. deserve to be read once more.

3D application

1. DO 2D GANS KNOW 3D SHAPE? UNSUPERVISED 3D SHAPE RECONSTRUCTION FROM 2D IMAGE GANS.. [github]
   • A fancy pipline in which takes gan inversion to reduce the artifacts of hand-crafted images, so that enabling training decompositional network.
   • They use discriminator's feature and l2 regularization term to constraint gan inversion step. Particularly, the form of l2 regularization is noticable.
   • They also realized that gan inversion cannot preserve all semantics of the original instance.

Tools

1. Learning Continuous Image Representation With Local Implicit Image Function. Nice work.