search

PeterouZh / CIPS-3D

3D-aware GANs based on NeRF (arXiv).
MIT License
610 stars 60 forks source link

> I want test some other image on your model. But I dont konw how to do it. If I have image sequence with pose data，how to test？ #33

Closed zhywanna closed 2 years ago

zhywanna commented 2 years ago

I want test some other image on your model. But I dont konw how to do it. If I have image sequence with pose data，how to test？

  1. Align the images in the way of StyleGAN. You can refer to this script align_images.py.
  2. Project the aligned images into the W space, also known as GAN inversion. Different from the common 2D inversion, you'd better set an appropriate yaw/pitch/fov for the CIPS-3D generator to make the initial pose of G(w) and the image to be inverted consistent.
  3. After you get the w of the image, you can reconstruct images of different styles using G'(w). G' can be obtained by interpolating generators of different domains.

Hope this helps.

Originally posted by @PeterouZh in https://github.com/PeterouZh/CIPS-3D/issues/7#issuecomment-963163677

zhywanna commented 2 years ago

Hi Zhou again, I'm currently studying and doing experiment on inverting 3D-aware-gan with optimization based method, and just found this earlier issue. Could you please show me some details about your step 2 ? Any advice will be really helpful for a beginner! Thanks a lot!

PeterouZh commented 2 years ago

Hi Zhou again, I'm currently studying and doing experiment on inverting 3D-aware-gan with optimization based method, and just found this earlier issue. Could you please show me some details about your step 2 ? Any advice will be really helpful for a beginner! Thanks a lot!

I think this paper will help you https://arxiv.org/abs/2102.07064.