EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks
Chan et al., CVPR 2022
Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge. Existing 3D GANs are either compute-intensive or make approximations that are not 3D-consistent; the former limits quality and resolution of the generated images and the latter adversely affects multi-view consistency and shape quality. In this work, we improve the computational efficiency and image quality of 3D GANs without overly relying on these approximations. We introduce an expressive hybrid explicit-implicit network architecture that, together with other design choices, synthesizes not only high-resolution multi-view-consistent images in real time but also produces high-quality 3D geometry. By decoupling feature generation and neural rendering, our framework is able to leverage state-of-the-art 2D CNN generators, such as StyleGAN2, and inherit their efficiency and expressiveness. We demonstrate state-of-the-art 3D-aware synthesis with FFHQ and AFHQ Cats, among other experiments.
🔑 Key idea:
Computational efficiency and image quality of 3D GANs exploiting StyleGAN2 (end-to-end training from random init) that generates tri-plane hybrid 3D representations.
💪 Strength:
Tri-plane hybrid 3D representation is efficient for neural rendering.
Dual discrimination with the final generated image and upsampled low-resolution generation image enhances the quality.
😵 Weakness:
Why not trilinear interpolation instead of bilinear interpolation and summation?
Runtime is around 30 fps. Better than some others, while GIRAFFE (Niemeyer and Geiger, 2021) is 180 fps. They said, "we believe major improvements in image quality, geometry quality, and view-consistency outweigh the increased compute cost." in Sec 5.1.
EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks
Chan et al., CVPR 2022
🔑 Key idea:
💪 Strength:
😵 Weakness:
🤔 Confidence: