search

SheldonTsui / GOF_NeurIPS2021

The codebase for our paper "Generative Occupancy Fields for 3D Surface-Aware Image Synthesis" (NeurIPS 2021)
Apache License 2.0
102 stars 6 forks source link

Access to the "correct outward-facing faces" #5

Closed MrTornado24 closed 2 years ago

MrTornado24 commented 2 years ago

Hi, thanks for your work and released code! Could you share the correct outward-facing faces so that we can train from scratch?

SheldonTsui commented 2 years ago

Hi, I have already provided them. Please refer to the pretrained models in README.

MrTornado24 commented 2 years ago

Thanks for your answer. What I mean is the "correct outward-facing " images that you train with, rather than the pretrained model.

SheldonTsui commented 2 years ago

The "correct outward-facing faces" means the learned shapes are outward-facing, not inward-facing. This phenomenon is discussed in Sec4.2 of this paper "Campari: Camera-aware decomposed generative neural radiance fields". For the training images, we just use 2D images from BFM, CelebA, or Cats as presented in the paper.

MrTornado24 commented 2 years ago

Thanks a lot. I am still confusing that if you just use the original dataset, what is the difference between training from an early pretrained model and "train from scratch"? What is the setup of the first 2k iterations to make sure that the correct outward-facing shape can be learned? Campari just states this problem but I am interested in how you solve it?

SheldonTsui commented 2 years ago

In the experiments, we find that the model trained on the original datasets sometimes learned "wrong" inward-facing shapes and sometimes learned "correct" outward-facing shapes. It's just like a random result. We don't find the control factor yet. However, we find that the model will keep the inward/outward property as the training goes, ie, the model will keep the correct outward-facing shape if starting from an outward-facing one. Therefore, in order to ensure the final correct outward-facing shape, the training of all models starts from an early pretrained model with the correct outward-facing faces.

MrTornado24 commented 2 years ago

Thanks for your detailed explanation!