search

elliottwu / unsup3d

(CVPR'20 Oral) Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild
MIT License
1.19k stars 194 forks source link

replace neural mesh renderer with pytorch3d #38

Closed YunjieYu closed 2 years ago

YunjieYu commented 2 years ago

Hi, wu!

I‘m sorry to disturb you again. Inspired from your repo, I just replace the neural renderer with pytorch3d. Specially, I choose MeshRenderer in pytorch3d.

I first eliminated the inconsistency between NMR and PyTorch3D coordinate systems by overriding PerspectiveCameras class in pytorch3d. However, I found that the noise for shape is so big that the network can not learn a reasonable face shape. Note that I have trained the model with 100 epochs. The figure below is a comparison between NMR and PyTorch3D.

NMR: 1

PyTorch3D: 2

RasterizationSettings is showed as below:

     # hard rasterization
     raster_settings = RasterizationSettings(
            image_size=self.image_size,
            blur_radius=0.0,
            faces_per_pixel=1,
            bin_size=0
        )

I noticed that you mentioned one can add a smoothing loss to the depth map for alleviating the noisy depth problem in #9. So can you provide the specific form of such a smoothing loss? I really don't have a clue about this.

Thank you very much!

YunjieYu commented 2 years ago

I close this issue because the problem has been solved.

abhishekgahlot commented 2 years ago

@YunjieYu Would you be able to share the code or branch where you successfully migrated to pytorch3d. I want to use unsup3d as well but on CPU only machine with a rendering animation.

nihaomiao commented 2 years ago

@YunjieYu Would you like to share your codes here? Thanks a lot!