facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.5k stars 1.28k forks source link

How to render a mesh like MAYA #1373

Open shoutOutYangJie opened 1 year ago

shoutOutYangJie commented 1 year ago

I have a full-head mesh, like this, which I use Maya to render it. 1667287082863

then I use Pytorch3d to render the mesh, but the result is bad, like this:

image

the rendered image has no eye hole because the triangle faces behind front head are also rendered and cover the hole. The reality is also worse than maya. I want to know to how to render in pytorch3d like maya. I use the following code:

     self.cameras = FoVPerspectiveCameras(device=device, R=R, T=T, znear=0.01,zfar=50,
                                        fov=fov * 180/np.pi)
        # print(f'camears: {self.cameras}')
        self.lights = PointLights(device=device, location=[[0.0, 0.0, 1e5]],
                             ambient_color=[[1, 1, 1]],
                             specular_color=[[0., 0., 0.]], diffuse_color=[[0., 0., 0.]])
        raster_settings = RasterizationSettings(
            image_size=img_size,
            blur_radius=0.0,
            faces_per_pixel=1,
            bin_size=0,
            max_faces_per_bin =1000
        )
        blend_params = blending.BlendParams(background_color=[0, 0, 0])
        self.renderer = MeshRenderer(
            rasterizer=MeshRasterizer(
                cameras=self.cameras,
                raster_settings=raster_settings
            ),
            shader=SoftPhongShader(
                device=device,
                cameras=self.cameras,
                lights=self.lights,
                blend_params=blend_params
            )
        )

face_tex = self.colors.unsqueeze(0) #  self.colors.unsqueeze(0) # torch.sigmoid(self.colors.unsqueeze(0)) # 1,n,3
face_color_tv = TexturesVertex(face_tex)
print(vps_RT.shape, self.faces.shape)
mesh = Meshes(vps_RT.unsqueeze(0), self.faces.unsqueeze(0), face_color_tv)
rendered_img = self.render(mesh)
return rendered_img

Here is my obj file. https://drive.google.com/file/d/1ViIA6-rrZqFOMuJDuWqngCjY7Cdtt860/view?usp=sharing

bottler commented 1 year ago

If the triangles are consistently defined in the mesh, then setting cull_backfaces=True in the rasterization settings may be enough to get the eye holes.

I don't know which other aspects you are concerned about, but one difference is that in PyTorch3D you are using full ambient white light (hence it looks so flat) but the Maya image has been rendered with more complicated lighting.